-- Logs begin at Thu 2022-10-06 11:41:27 UTC, end at Thu 2022-10-06 12:52:07 UTC. -- Oct 06 12:07:02.088670 dv-cinder-856041-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 06 12:07:02.097710 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 06 12:07:02.099724 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Oct 6 12:07:02 2022] *** Oct 06 12:07:02.099724 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 06 12:07:02.099724 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 06 12:07:02.099724 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: nodename: dv-cinder-856041-2-iscsi Oct 06 12:07:02.099724 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: machine: x86_64 Oct 06 12:07:02.099724 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: clock source: unix Oct 06 12:07:02.099724 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: pcre jit disabled Oct 06 12:07:02.099724 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: detected number of CPU cores: 4 Oct 06 12:07:02.099724 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: current working directory: / Oct 06 12:07:02.099724 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: detected binary path: /usr/bin/uwsgi-core Oct 06 12:07:02.100380 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: your processes number limit is 47680 Oct 06 12:07:02.100380 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: your memory page size is 4096 bytes Oct 06 12:07:02.100380 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: detected max file descriptor number: 2048 Oct 06 12:07:02.100380 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: lock engine: pthread robust mutexes Oct 06 12:07:02.100380 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: thunder lock: enabled Oct 06 12:07:02.100380 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 06 12:07:02.100380 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 06 12:07:02.138997 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: Python main interpreter initialized at 0x56169c8589c0 Oct 06 12:07:02.138997 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: python threads support enabled Oct 06 12:07:02.138997 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: your server socket listen backlog is limited to 100 connections Oct 06 12:07:02.138997 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: your mercy for graceful operations on workers is 90 seconds Oct 06 12:07:02.139333 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: mapped 403077 bytes (393 KB) for 2 cores Oct 06 12:07:02.139426 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: *** Operational MODE: preforking *** Oct 06 12:07:02.139696 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: *** uWSGI is running in multiple interpreter mode *** Oct 06 12:07:02.139696 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: spawned uWSGI master process (pid: 81343) Oct 06 12:07:02.139892 dv-cinder-856041-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 06 12:07:02.143535 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: spawned uWSGI worker 1 (pid: 81354, cores: 1) Oct 06 12:07:02.143535 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: spawned uWSGI worker 2 (pid: 81355, cores: 1) Oct 06 12:07:02.143535 dv-cinder-856041-2-iscsi devstack@keystone.service[81343]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 06 12:07:06.405743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 06 12:07:06.405743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: from cryptography.utils import int_from_bytes Oct 06 12:07:06.406526 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 06 12:07:06.406526 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: from cryptography.utils import int_from_bytes Oct 06 12:07:06.406980 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 06 12:07:06.406980 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: from cryptography.utils import int_from_bytes Oct 06 12:07:06.407120 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 06 12:07:06.407120 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: from cryptography.utils import int_from_bytes Oct 06 12:07:06.456534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 06 12:07:06.456685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 06 12:07:06.456778 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 06 12:07:06.456854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 06 12:07:06.456922 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] command line args: [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 06 12:07:06.457059 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] command line args: [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 06 12:07:06.457153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 06 12:07:06.457319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 06 12:07:06.457406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ================================================================================ {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 06 12:07:06.457486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ================================================================================ {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 06 12:07:06.457558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] admin_token = **** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.457738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] admin_token = **** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.457831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] config_dir = [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.457980 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.458067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] config_dir = [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.458171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] config_source = [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.458251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.458356 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] config_source = [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.458447 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] debug = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.458523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] debug = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.458594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.458831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.458967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.459068 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.459174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.459277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.459417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.459565 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.459688 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.459787 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.459888 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] list_limit = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.459985 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] list_limit = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.460087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] log_config_append = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.460185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] log_config_append = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.460295 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.460400 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.460497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] log_dir = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.460591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] log_dir = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.460699 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] log_file = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.460802 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] log_file = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.460927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.461007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.461126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.461201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.461324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.461401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.461530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.461612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.461733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.461817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.461890 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.462017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.462095 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.462212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.462289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.462413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.462500 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.462631 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.462709 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.462782 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.462847 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.462918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.463040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.463130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.463288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.463384 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.463639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.463756 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.463918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.464018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.464429 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.464514 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.464632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] publish_errors = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.464762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] publish_errors = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.464911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.464994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.465074 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.465201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.465279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.465358 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.465483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.465579 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.465662 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.465736 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.465898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] standard_threads = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.465987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] standard_threads = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.466127 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.466213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.466368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.466454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.466528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.466598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.466706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] use_journal = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.466837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] use_journal = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.466914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] use_json = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.466990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] use_json = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.467064 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] use_stderr = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.467183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] use_stderr = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.467260 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] use_syslog = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.467342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] use_syslog = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.467422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.467495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 12:07:06.467696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.467786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.467934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.468027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.468191 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.468269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.468426 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.468509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.468672 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.468759 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.468909 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.468989 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.469154 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.469238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.469386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.469513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.469631 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.469709 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.469875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.469954 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.470122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.470212 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.470351 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.470429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.470594 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.470678 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.470826 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.470913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.471070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.471148 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.471314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.471405 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.471568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.471672 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.471824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.471909 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.472075 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.472153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.472314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.472475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.472557 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.472629 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.472787 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.472867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.473045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.473129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.473295 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.473373 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.473542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.473621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.473771 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.473851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.473998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.474083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.474225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.474303 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.474454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.474534 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.474700 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.474784 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.474944 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.475027 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.475171 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.475251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.475397 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.475474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.475664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.475743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.475908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.475994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.476143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.476221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.476388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.476465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.476615 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.476700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.476845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.476923 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.477077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.477155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.477302 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.477381 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.477528 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.477606 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.477753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.477832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.477980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.478055 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.478201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.478288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.478444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.478528 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.478670 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.478749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.478902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.478980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.479129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.479214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.479359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.479439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.479613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.479694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.479860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.479937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.480612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.480721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.480903 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.481010 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.481140 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.481245 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.481360 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.481459 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.481597 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.481719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.481833 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.481922 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.482060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.482145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.482307 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.482407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.482538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.482624 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.482771 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.482856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.482999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.483135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.483258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.483390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.483476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.483598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.483728 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.483806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.483965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.484055 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.484183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.484269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.484455 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.484532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.484692 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.484776 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.484941 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.485027 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.485183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.485268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.485417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.485507 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.485647 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.485735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.485878 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.485964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.486124 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.486212 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.486360 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.486454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.486595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.486717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.486834 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.486919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.487067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.487154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.487311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.487435 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.487566 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.487654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.487790 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.487879 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.488023 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.488102 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.488256 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.488334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.488488 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.488575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.488750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.488852 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.488989 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.489077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.489234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.489324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.489472 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.489548 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.489708 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.489793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.489950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.490116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.490198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.490271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.490404 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.490501 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.490637 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.490726 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.490863 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.490944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.491090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.491169 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.491316 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.491402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.491565 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.491650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.491796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.491876 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.492032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.492133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.492252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.492331 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.492538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.492625 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.492733 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.492813 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.492985 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.493066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.493242 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.493321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.493478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.493589 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.493706 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] auth.external = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.493787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] auth.external = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.493933 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.494014 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.494225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.494305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.494455 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.494543 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.494683 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] auth.password = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.494769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] auth.password = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.494908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] auth.token = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.494989 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] auth.token = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.495143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.495222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.495373 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.495455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.495675 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.496275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.496434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.496526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.496668 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.496748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.496884 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.496966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.497114 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.497194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.497347 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] credential.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.497428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] credential.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.497594 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.497679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.497856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.497937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.498086 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.498174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.498325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.498405 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.498566 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.498646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.498796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.498876 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.499039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.499119 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.499291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.499370 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.499589 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.499714 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.499798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.499900 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.500063 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.500146 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.500306 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.500388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.500565 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.500644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.500803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.500882 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.501058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.501141 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.501288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] federation.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.501368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] federation.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.501519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.501599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.501762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.501843 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.502005 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.502088 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.502230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.502310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.502474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.502555 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.502706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.502786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.502956 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.503044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.503188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.503268 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.503438 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.503544 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.503697 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.503789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.503943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.504022 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.504173 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.504252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.504417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.504574 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.504652 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.504724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.504883 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.504964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.505111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.505199 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.505357 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.505436 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.505585 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.505664 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.505820 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.505911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.506066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.506146 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.506314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.506395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.506544 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.506631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.506771 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.506851 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.507000 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.507079 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.507241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.507342 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.507488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.507609 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.507758 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.507840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.508014 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.508112 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.508259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.508339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.508527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.508607 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.508765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.508856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.509002 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.509083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.509231 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.509311 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.509466 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.509554 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.509692 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.509813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.509924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.510005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.510163 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.510252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.510401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.510481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.510648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.510728 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.510875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.510963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.511120 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.511201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.511387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.511467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.511643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.511736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.511889 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.511969 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.512132 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.512212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.512364 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.512462 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.512594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.512674 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.512820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.512900 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.513053 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.513136 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.513286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.513367 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.513523 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.513603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.513757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.513839 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.514098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.514178 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.514351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.514431 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.514599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.514680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.514828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.514908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.515057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.515143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.515307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.515394 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.515577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.515665 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.515817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.515898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.516052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.516134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.516285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.516366 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.516514 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.516594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.516751 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.516833 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.517010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.517091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.517241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.517321 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.517489 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.517570 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.517736 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.517816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.517983 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.518063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.518216 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.518367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.518447 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.518520 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.518663 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.518744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.518893 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.518981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.519127 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.519214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.519355 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.519434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.519622 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.519704 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.519869 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.519956 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.520114 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.520195 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.520360 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.520443 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.520604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.520684 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.520830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.520910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.521074 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.521155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.521309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.521389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.521542 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.521622 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.521775 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.521857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.522008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.522088 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.522251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.522331 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.522498 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.522579 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.522732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.522812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.522986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.523071 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.523214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.523296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.523455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.523590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.523716 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.523797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.523949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.524030 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.524184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.524264 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.524428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.524508 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.524668 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.524749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.524896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.524976 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.525128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.525207 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.525364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] resource.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.525453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] resource.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.525611 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.525691 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.525859 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.525938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.526091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.526172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.526338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.526418 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.526587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.526667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.526820 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.527396 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.527616 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.527701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.527857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.528019 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.528098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.528170 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.528316 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] role.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.528397 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] role.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.528562 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] role.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.528650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] role.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.528789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.528873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.529037 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.529116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.529282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.529371 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.529529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.529610 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.529772 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.529852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.530268 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.530359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.530513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.530593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.530759 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.530838 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.531003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.531092 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.531234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.531314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.531480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.531630 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.531750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.531846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.531996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.532077 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.532241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.532321 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.532635 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.532724 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.532878 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.532984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.533122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.533203 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.533372 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.533462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.533621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.533700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.533867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.533945 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.534099 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.534179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.534352 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.534430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.534604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.534681 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.534840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.534919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.535069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.535148 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.535297 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.535375 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.535546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.535630 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.535804 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.535884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.536059 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.536137 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.536304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.536385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.536539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.536617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.536771 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.536849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.537001 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.537081 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.537233 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] token.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.537312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] token.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.537468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.537546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.537715 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.537794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.537950 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.538031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.538204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.538282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.538449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.538529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.538685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.538763 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.538928 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.539007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.539170 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.539250 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.539416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.539494 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.539676 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.539755 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.539914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.539995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.540147 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.540225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.540392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.540469 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.540667 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.540752 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.540896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.540982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.541134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.541221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.541395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.541473 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.541652 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.connection = **** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.541737 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.connection = **** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.541893 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.541971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.542138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.542216 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.542374 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.542460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.543175 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.543263 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.543416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.543519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.543675 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.543766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.543911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.543990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.544144 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.544221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.544388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.544474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.544622 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.544700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.544865 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.544946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.545096 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.545185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.545343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.545421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.545576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.545653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.545816 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.545905 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.546044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.546121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.546280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.546358 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.546511 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.546596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 12:07:06.546670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 06 12:07:06.546741 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 06 12:07:06.575547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.575680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.575763 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.575841 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.575964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.576049 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.584315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.584493 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.613571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.613699 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.614303 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.614380 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.668300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.668652 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.716799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.717037 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.717725 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.717964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.873252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.873420 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.873494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.873570 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.873695 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.873802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.873918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.874001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.874138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.874220 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.874357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.874441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.897243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.897483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.897733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.897816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.897957 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.898041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.898182 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.898310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.898395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.898489 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.898595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.898727 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.898807 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.899013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.910563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.915039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.923648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.923887 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.924114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.924337 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.924556 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.924772 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.924988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.926417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.926655 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.926885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.927109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.927326 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.927569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.927789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 12:07:06.941631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.941842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 06 12:07:06.942170 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.942253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.942372 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 06 12:07:06.942453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 06 12:07:06.942649 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.942732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.942897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 06 12:07:06.942978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.943165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.943254 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.943423 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.943532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.943711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.943978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.953790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.954075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.954164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.954351 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.954435 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.954639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.954731 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.954926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.961599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:06.961916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:06.965059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:06.965354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.965437 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:06.965640 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.965723 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.965926 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.966010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.966215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.966298 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.966481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.966563 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.966753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.966845 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.967030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.967113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.967303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.967383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.967605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.967688 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.967882 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.967962 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.968153 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.990471 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.990715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.991963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:06.992229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:06.992313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.992519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.996932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:06.997224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.997306 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:06.997510 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.997593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:06.997795 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.003936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.004151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.004235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.004442 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.009813 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.010067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.010150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.010368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.010452 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.010662 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.010753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.010936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.011017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.011214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.011803 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.012027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.012111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.012300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.012382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.012574 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.028290 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.028570 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.028654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.028931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.032705 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.033070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.037577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.037577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.038692 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.038692 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.040454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.040826 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.043647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.044057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.046886 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.047186 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.047284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.047479 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.047612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.047792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.047905 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.048072 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.048193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.048364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.048445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.048734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.064524 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.064621 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.064815 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.064897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.065086 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.065203 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.065348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.065432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.072191 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.072280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.073728 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.074361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.075820 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.075908 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.076102 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.076184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.078759 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.078853 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.079048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.079130 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.079333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.079414 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.079644 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.079729 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.080046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.080140 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.091691 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.091805 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.091987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.092070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.092274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.092366 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.092559 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.092641 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.092939 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.093020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.093223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.093312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.093503 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.093584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.108899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.109001 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.111830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.111935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.114707 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.114858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.116172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.116271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.117679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.117761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.117968 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.118051 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.118380 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.118507 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.124825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.124918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.126244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.126326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.129155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.129241 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.129554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.129635 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.129843 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.129924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.130124 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.130207 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.138442 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.138534 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.138739 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.138834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.139023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.139103 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.145921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.146013 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 12:07:07.146209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.146290 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.146491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.146572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.146770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.146850 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.147041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.147122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.147312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.147394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.147609 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.147704 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.147882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.147971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.148165 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.148253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.148445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.148526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.148724 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.148803 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.148990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.149074 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.149256 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.149336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.149542 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.149623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 12:07:07.184205 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56169c8589c0 pid: 81354 (default app) Oct 06 12:07:07.184409 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56169c8589c0 pid: 81355 (default app) Oct 06 12:07:07.187438 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72b433-fe06-4b73-93e5-8e21c381af23 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:07.187751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72b433-fe06-4b73-93e5-8e21c381af23 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:07.188028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72b433-fe06-4b73-93e5-8e21c381af23 None None] PATH_INFO: `/v3` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:07.192210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Thu Oct 6 12:07:07 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:13.984329 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74eac2-879f-4dea-96e7-a47416ef0a97 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:13.984842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74eac2-879f-4dea-96e7-a47416ef0a97 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:13.984944 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74eac2-879f-4dea-96e7-a47416ef0a97 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:13.989502 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:07:13.999207 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30effc17-25f4-4f7b-b65b-2534ed9ababb None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:13.999207 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30effc17-25f4-4f7b-b65b-2534ed9ababb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:13.999207 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30effc17-25f4-4f7b-b65b-2534ed9ababb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:14.067661 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: INFO dbcounter [None req-30effc17-25f4-4f7b-b65b-2534ed9ababb None None] Registered counter for database keystone Oct 06 12:07:14.086710 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG oslo_db.sqlalchemy.engines [None req-30effc17-25f4-4f7b-b65b-2534ed9ababb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81354) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 06 12:07:14.093642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writer thread running {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 06 12:07:14.272138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG passlib.handlers.bcrypt [None req-30effc17-25f4-4f7b-b65b-2534ed9ababb None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81354) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 06 12:07:14.272526 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG passlib.handlers.bcrypt [None req-30effc17-25f4-4f7b-b65b-2534ed9ababb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81354) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 06 12:07:14.330093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-30effc17-25f4-4f7b-b65b-2534ed9ababb None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:14.331738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-30effc17-25f4-4f7b-b65b-2534ed9ababb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:14.456553 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 06 12:07:14.905080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74eac2-879f-4dea-96e7-a47416ef0a97 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:14.974614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: INFO dbcounter [None req-fd74eac2-879f-4dea-96e7-a47416ef0a97 None None] Registered counter for database keystone Oct 06 12:07:14.996483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG oslo_db.sqlalchemy.engines [None req-fd74eac2-879f-4dea-96e7-a47416ef0a97 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81355) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 06 12:07:15.002700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writer thread running {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 06 12:07:15.119725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74eac2-879f-4dea-96e7-a47416ef0a97 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:15.124026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b74109-967c-4d18-addb-e0d897d0956d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:15.124673 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b74109-967c-4d18-addb-e0d897d0956d admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:15.124954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b74109-967c-4d18-addb-e0d897d0956d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:15.125228 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b74109-967c-4d18-addb-e0d897d0956d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:15.133341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b74109-967c-4d18-addb-e0d897d0956d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:15.185064 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b3b74109-967c-4d18-addb-e0d897d0956d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 06 12:07:15.185345 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b74109-967c-4d18-addb-e0d897d0956d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:15.194865 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Thu Oct 6 12:07:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 12:07:15.200931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30effc17-25f4-4f7b-b65b-2534ed9ababb None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:15.218195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30effc17-25f4-4f7b-b65b-2534ed9ababb None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:15.222667 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02279e-d3ec-4360-9804-5dafbc339d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:15.223794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02279e-d3ec-4360-9804-5dafbc339d67 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:15.224110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02279e-d3ec-4360-9804-5dafbc339d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:15.224384 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02279e-d3ec-4360-9804-5dafbc339d67 admin admin] PATH_INFO: `/v3/projects/5568e23a42b74b0a8d5f679d197b7521` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:15.225969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02279e-d3ec-4360-9804-5dafbc339d67 admin admin] RBAC: Authorizing `identity:get_project(project_id=5568e23a42b74b0a8d5f679d197b7521)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:15.278678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ba02279e-d3ec-4360-9804-5dafbc339d67 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 06 12:07:15.279038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02279e-d3ec-4360-9804-5dafbc339d67 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:15.284047 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Thu Oct 6 12:07:15 2022] GET /identity/v3/projects/5568e23a42b74b0a8d5f679d197b7521 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:17.404367 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3da3d-25e3-404f-b19a-7777d361ef89 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:17.404880 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3da3d-25e3-404f-b19a-7777d361ef89 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:17.404966 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3da3d-25e3-404f-b19a-7777d361ef89 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:17.409292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:17.414740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b58ec4-bc9c-472f-83e9-3269013a1c93 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:17.415107 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b58ec4-bc9c-472f-83e9-3269013a1c93 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:17.415295 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b58ec4-bc9c-472f-83e9-3269013a1c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:17.435980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-11b58ec4-bc9c-472f-83e9-3269013a1c93 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:17.437490 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-11b58ec4-bc9c-472f-83e9-3269013a1c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:17.445284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 06 12:07:17.714566 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3da3d-25e3-404f-b19a-7777d361ef89 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:17.726757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3da3d-25e3-404f-b19a-7777d361ef89 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:17.731062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d34d6-ba37-4351-983b-63199fedcd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:17.731686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d34d6-ba37-4351-983b-63199fedcd59 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:17.731974 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d34d6-ba37-4351-983b-63199fedcd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:17.732251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d34d6-ba37-4351-983b-63199fedcd59 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:17.739832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d34d6-ba37-4351-983b-63199fedcd59 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:17.741605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d34d6-ba37-4351-983b-63199fedcd59 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:17.750855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Thu Oct 6 12:07:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 12:07:17.756029 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b58ec4-bc9c-472f-83e9-3269013a1c93 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:17.765509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b58ec4-bc9c-472f-83e9-3269013a1c93 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:17.769784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4575809-7bbb-4bec-b70f-135d4a28c06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:17.770401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4575809-7bbb-4bec-b70f-135d4a28c06b admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:17.770680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4575809-7bbb-4bec-b70f-135d4a28c06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:17.770951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4575809-7bbb-4bec-b70f-135d4a28c06b admin admin] PATH_INFO: `/v3/users/50b3556af8274b8a9a59fe39e12271c7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:17.772274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4575809-7bbb-4bec-b70f-135d4a28c06b admin admin] RBAC: Authorizing `identity:get_user(user_id=50b3556af8274b8a9a59fe39e12271c7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:17.774082 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4575809-7bbb-4bec-b70f-135d4a28c06b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:17.775591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Thu Oct 6 12:07:17 2022] GET /identity/v3/users/50b3556af8274b8a9a59fe39e12271c7 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:19.976818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0568a-013c-4f96-8baf-d1f3ef0752de None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:19.977389 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0568a-013c-4f96-8baf-d1f3ef0752de None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:19.977389 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0568a-013c-4f96-8baf-d1f3ef0752de None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:19.978696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:19.984501 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e3cdc-c34e-48a3-ac0e-ab9085091fb4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:19.984932 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e3cdc-c34e-48a3-ac0e-ab9085091fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:19.985167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e3cdc-c34e-48a3-ac0e-ab9085091fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:19.993537 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ed7b0-154b-4dc6-9548-198f499fc529 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:19.993825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ed7b0-154b-4dc6-9548-198f499fc529 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:19.994100 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ed7b0-154b-4dc6-9548-198f499fc529 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:19.995375 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:07:20.000483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d4efd-191d-4552-ab37-18a9fc8d1dcb None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:20.000770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d4efd-191d-4552-ab37-18a9fc8d1dcb None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:20.001048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d4efd-191d-4552-ab37-18a9fc8d1dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:20.003457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-676e3cdc-c34e-48a3-ac0e-ab9085091fb4 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:20.004674 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-676e3cdc-c34e-48a3-ac0e-ab9085091fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:20.043834 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG passlib.handlers.bcrypt [None req-2f9d4efd-191d-4552-ab37-18a9fc8d1dcb None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81355) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 06 12:07:20.044215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG passlib.handlers.bcrypt [None req-2f9d4efd-191d-4552-ab37-18a9fc8d1dcb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81355) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 06 12:07:20.086603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:07:20.093647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-2f9d4efd-191d-4552-ab37-18a9fc8d1dcb None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:20.095249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2f9d4efd-191d-4552-ab37-18a9fc8d1dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:20.195067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:07:20.355783 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e3cdc-c34e-48a3-ac0e-ab9085091fb4 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:20.380280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:20.460329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e3cdc-c34e-48a3-ac0e-ab9085091fb4 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:20.470220 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d4efd-191d-4552-ab37-18a9fc8d1dcb None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:20.493525 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d431d3-05c7-4c14-9c43-7650cbf49847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:20.494096 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d431d3-05c7-4c14-9c43-7650cbf49847 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:20.494376 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d431d3-05c7-4c14-9c43-7650cbf49847 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:20.494650 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d431d3-05c7-4c14-9c43-7650cbf49847 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:20.506027 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d431d3-05c7-4c14-9c43-7650cbf49847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:20.515332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-20d431d3-05c7-4c14-9c43-7650cbf49847 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:07:20.515332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:20.515332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:20.515332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:20.515332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:20.515332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:20.515332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:20.515332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:20.515332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:20.515332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:20.515332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:20.515332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:20.515332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:20.515332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:20.515332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:20.515332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:20.516395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:20.516395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:20.516395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:20.516395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:20.516395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:20.516395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:20.516395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:20.516395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:20.516395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:20.516395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:20.516395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:20.516395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:20.516395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:20.516395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:20.516395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:20.516395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:20.517646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:20.517646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:20.517646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:20.517646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:20.517646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:20.517646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:20.517646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:20.517646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:20.517646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:20.517646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:20.517646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:07:20.517646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:07:20.517646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:07:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:20.522153 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d431d3-05c7-4c14-9c43-7650cbf49847 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:20.575898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d4efd-191d-4552-ab37-18a9fc8d1dcb None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:20.578188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:20.604477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:20.613153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1811e96-c76f-4516-9b7c-3e39c8d6a4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:20.614019 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1811e96-c76f-4516-9b7c-3e39c8d6a4af None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:20.614298 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1811e96-c76f-4516-9b7c-3e39c8d6a4af None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:20.614579 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1811e96-c76f-4516-9b7c-3e39c8d6a4af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:20.623295 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d431d3-05c7-4c14-9c43-7650cbf49847 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:20.656656 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086719a-c04a-4e04-b428-6131c606b133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:20.657252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086719a-c04a-4e04-b428-6131c606b133 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:20.657539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086719a-c04a-4e04-b428-6131c606b133 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:20.657814 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086719a-c04a-4e04-b428-6131c606b133 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:20.658482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086719a-c04a-4e04-b428-6131c606b133 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:20.658787 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086719a-c04a-4e04-b428-6131c606b133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:20.660227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086719a-c04a-4e04-b428-6131c606b133 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:20.671640 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:07:20.677179 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086719a-c04a-4e04-b428-6131c606b133 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:20.681933 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1811e96-c76f-4516-9b7c-3e39c8d6a4af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:20.684013 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1811e96-c76f-4516-9b7c-3e39c8d6a4af None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:20.775449 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086719a-c04a-4e04-b428-6131c606b133 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:20.781106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 8/16] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:07:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:07:20.786783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1811e96-c76f-4516-9b7c-3e39c8d6a4af None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:20.789583 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:20.812725 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15b89b-fc69-4dc4-81fa-56788816b20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:20.813364 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15b89b-fc69-4dc4-81fa-56788816b20a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:20.813650 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15b89b-fc69-4dc4-81fa-56788816b20a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:20.813929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15b89b-fc69-4dc4-81fa-56788816b20a None admin] PATH_INFO: `/v3/users/50b3556af8274b8a9a59fe39e12271c7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:20.815076 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15b89b-fc69-4dc4-81fa-56788816b20a None admin] RBAC: Authorizing `identity:get_user(user_id=50b3556af8274b8a9a59fe39e12271c7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:20.816571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15b89b-fc69-4dc4-81fa-56788816b20a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:20.818164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:20 2022] GET /identity/v3/users/50b3556af8274b8a9a59fe39e12271c7 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:20.824264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15b89b-fc69-4dc4-81fa-56788816b20a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:20.838994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:20.886375 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1811e96-c76f-4516-9b7c-3e39c8d6a4af None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:20.920585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd71321-e47b-4bf5-b7c9-eb0ca9028f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:20.921443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd71321-e47b-4bf5-b7c9-eb0ca9028f4c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:20.921725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd71321-e47b-4bf5-b7c9-eb0ca9028f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:20.922000 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd71321-e47b-4bf5-b7c9-eb0ca9028f4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:20.923935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15b89b-fc69-4dc4-81fa-56788816b20a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:20.941004 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd71321-e47b-4bf5-b7c9-eb0ca9028f4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:20.942468 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd71321-e47b-4bf5-b7c9-eb0ca9028f4c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:20.958992 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06dad3-ab3e-4c58-836d-8a3bdd2c9ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:20.959699 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06dad3-ab3e-4c58-836d-8a3bdd2c9ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:20.959984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06dad3-ab3e-4c58-836d-8a3bdd2c9ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:20.960283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06dad3-ab3e-4c58-836d-8a3bdd2c9ad3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:20.960437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-6fd71321-e47b-4bf5-b7c9-eb0ca9028f4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:07:20.960437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:20.960437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:20.960437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:20.960437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:20.960437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:20.960437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:20.960437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:20.960437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:20.960437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:20.960437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:20.960437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:20.960437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:07:20.960437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:07:20.960437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:07:20.960437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:20.961441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:07:20.962738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:07:20.962738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:07:20.962738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:07:20.962738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 9/18] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:07:20.963002 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06dad3-ab3e-4c58-836d-8a3bdd2c9ad3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:20.964401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06dad3-ab3e-4c58-836d-8a3bdd2c9ad3 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:20.965979 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 10/19] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:07:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:07:20.968048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd71321-e47b-4bf5-b7c9-eb0ca9028f4c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:20.971428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06dad3-ab3e-4c58-836d-8a3bdd2c9ad3 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:21.024616 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:21.050278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:21.069496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06dad3-ab3e-4c58-836d-8a3bdd2c9ad3 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:21.070520 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd71321-e47b-4bf5-b7c9-eb0ca9028f4c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:21.102924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f30ce-8d30-4c46-9d3d-4d67462e7c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:21.103738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f30ce-8d30-4c46-9d3d-4d67462e7c24 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:21.104169 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f30ce-8d30-4c46-9d3d-4d67462e7c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:21.104362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03234e43-3ae8-4980-a942-c07d2f11a3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:21.104488 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f30ce-8d30-4c46-9d3d-4d67462e7c24 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:21.105460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03234e43-3ae8-4980-a942-c07d2f11a3a3 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:21.105589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f30ce-8d30-4c46-9d3d-4d67462e7c24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:21.105738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03234e43-3ae8-4980-a942-c07d2f11a3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:21.105893 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f30ce-8d30-4c46-9d3d-4d67462e7c24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:21.106035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03234e43-3ae8-4980-a942-c07d2f11a3a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:21.106749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03234e43-3ae8-4980-a942-c07d2f11a3a3 None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.domain.id=default, user.id=50b3556af8274b8a9a59fe39e12271c7) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:21.107053 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03234e43-3ae8-4980-a942-c07d2f11a3a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:21.107264 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f30ce-8d30-4c46-9d3d-4d67462e7c24 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:21.108527 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03234e43-3ae8-4980-a942-c07d2f11a3a3 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:21.114966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 11/20] 10.222.0.33 () {56 vars in 1322 bytes} [Thu Oct 6 12:07:20 2022] GET /identity/v3/role_assignments?user.id=50b3556af8274b8a9a59fe39e12271c7&scope.domain.id=default&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:07:21.125797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 10/21] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:23.638920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56f1fa-f1c4-480a-ae30-cfd70e211fa1 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:23.639451 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56f1fa-f1c4-480a-ae30-cfd70e211fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:23.639451 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56f1fa-f1c4-480a-ae30-cfd70e211fa1 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:23.641115 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 12/22] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:23.646430 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79820f-16bc-49ba-97b6-107a8aacc856 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:23.647255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79820f-16bc-49ba-97b6-107a8aacc856 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:23.647849 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79820f-16bc-49ba-97b6-107a8aacc856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:23.675632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-4c79820f-16bc-49ba-97b6-107a8aacc856 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:23.677176 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4c79820f-16bc-49ba-97b6-107a8aacc856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:23.769331 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 11/23] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:23.893440 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b1a45-fc60-481e-be23-73c51b672ab4 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:23.893622 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b1a45-fc60-481e-be23-73c51b672ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:23.893902 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b1a45-fc60-481e-be23-73c51b672ab4 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:23.895556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 13/24] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:23.900766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dec43d-256c-4c76-b664-2bc653bc5216 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:23.901298 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dec43d-256c-4c76-b664-2bc653bc5216 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:23.902399 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dec43d-256c-4c76-b664-2bc653bc5216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:23.926737 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b0dec43d-256c-4c76-b664-2bc653bc5216 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:23.928470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b0dec43d-256c-4c76-b664-2bc653bc5216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:23.982084 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:24.014571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 12/25] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:24.169747 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a1d7e-cd74-4f00-b547-ef2f7b9f61ec None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:24.170089 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a1d7e-cd74-4f00-b547-ef2f7b9f61ec None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:24.170288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a1d7e-cd74-4f00-b547-ef2f7b9f61ec None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:24.171592 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 14/26] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:24.176905 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f8e64-42de-4295-932e-52b2ac54f620 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:24.177199 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f8e64-42de-4295-932e-52b2ac54f620 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:24.177476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f8e64-42de-4295-932e-52b2ac54f620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:24.204513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-189f8e64-42de-4295-932e-52b2ac54f620 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:24.205861 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-189f8e64-42de-4295-932e-52b2ac54f620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:24.221881 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cefb02-9cd7-46ea-a6b4-1ca64ca6e746 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:24.222191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cefb02-9cd7-46ea-a6b4-1ca64ca6e746 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:24.222441 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cefb02-9cd7-46ea-a6b4-1ca64ca6e746 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:24.223994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 15/27] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:07:24.233875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cce241-604b-4962-8c73-4a13121efcc8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:24.234164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cce241-604b-4962-8c73-4a13121efcc8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:24.234444 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cce241-604b-4962-8c73-4a13121efcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:24.268761 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d2cce241-604b-4962-8c73-4a13121efcc8 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:24.270610 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d2cce241-604b-4962-8c73-4a13121efcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:24.358801 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 16/28] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:07:24.359530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cce241-604b-4962-8c73-4a13121efcc8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:24.369358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 13/29] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:24.459282 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:24.492968 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cce241-604b-4962-8c73-4a13121efcc8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:24.532604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89b593-7d76-41cc-9d8f-747da6238f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:24.533235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89b593-7d76-41cc-9d8f-747da6238f61 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:24.533512 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89b593-7d76-41cc-9d8f-747da6238f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:24.533779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89b593-7d76-41cc-9d8f-747da6238f61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:24.540577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f8e64-42de-4295-932e-52b2ac54f620 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:24.549381 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89b593-7d76-41cc-9d8f-747da6238f61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:24.554721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-ea89b593-7d76-41cc-9d8f-747da6238f61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:07:24.554721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:24.554721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:24.554721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:24.554721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:24.554721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:24.554721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:24.554721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:24.554721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:24.554721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:24.554721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:24.554721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:24.554721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:24.554721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:24.554721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:24.554721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:24.555629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:24.555629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:24.555629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:24.555629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:24.555629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:24.555629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:24.555629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:24.555629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:24.555629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:24.555629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:24.555629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:24.555629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:24.555629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:24.555629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:24.555629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:24.555629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:24.556408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:24.556408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:24.556408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:24.556408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:24.556408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:24.556408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:24.556408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:24.556408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:24.556408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:24.556408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:24.556408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:07:24.556408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:07:24.556408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 17/30] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:07:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:24.564227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89b593-7d76-41cc-9d8f-747da6238f61 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:24.717245 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f8e64-42de-4295-932e-52b2ac54f620 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:24.726387 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:24.755033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89b593-7d76-41cc-9d8f-747da6238f61 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:24.769405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8eebe-f0d9-4885-99c9-2a8b6ce829d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:24.770358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8eebe-f0d9-4885-99c9-2a8b6ce829d3 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:24.770896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8eebe-f0d9-4885-99c9-2a8b6ce829d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:24.771467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8eebe-f0d9-4885-99c9-2a8b6ce829d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:24.785592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8eebe-f0d9-4885-99c9-2a8b6ce829d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:24.787186 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8eebe-f0d9-4885-99c9-2a8b6ce829d3 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:24.803418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-bca8eebe-f0d9-4885-99c9-2a8b6ce829d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:07:24.803418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:24.803418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:24.803418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:24.803418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:24.803418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:24.803418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:24.803418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:24.803418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:24.803418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:24.803418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:24.803418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:24.803418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:07:24.803418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:07:24.803418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:07:24.803418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:24.804505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:07:24.805320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:07:24.805320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:07:24.805320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:07:24.805320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 14/31] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:24.810571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8eebe-f0d9-4885-99c9-2a8b6ce829d3 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:24.810827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf3fd2-cbc6-4514-9c08-8fd577e2512d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:24.812077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf3fd2-cbc6-4514-9c08-8fd577e2512d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:24.812670 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf3fd2-cbc6-4514-9c08-8fd577e2512d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:24.813193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf3fd2-cbc6-4514-9c08-8fd577e2512d None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:24.814276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf3fd2-cbc6-4514-9c08-8fd577e2512d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:24.814861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf3fd2-cbc6-4514-9c08-8fd577e2512d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:24.816682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf3fd2-cbc6-4514-9c08-8fd577e2512d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:24.818653 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:24.829782 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 18/32] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:24.831021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf3fd2-cbc6-4514-9c08-8fd577e2512d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:24.915768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8eebe-f0d9-4885-99c9-2a8b6ce829d3 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:24.943376 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf3fd2-cbc6-4514-9c08-8fd577e2512d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:24.950779 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a127c5-6d42-4e9c-8730-b3d11cacd9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:24.951382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a127c5-6d42-4e9c-8730-b3d11cacd9ea None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:24.951688 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a127c5-6d42-4e9c-8730-b3d11cacd9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:24.951959 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a127c5-6d42-4e9c-8730-b3d11cacd9ea None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:24.952620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a127c5-6d42-4e9c-8730-b3d11cacd9ea None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:24.953990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a127c5-6d42-4e9c-8730-b3d11cacd9ea None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:24.982177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6bf8b8-7323-4d65-9146-7af562cab61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:24.983107 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6bf8b8-7323-4d65-9146-7af562cab61d None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:24.983625 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6bf8b8-7323-4d65-9146-7af562cab61d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:24.984100 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6bf8b8-7323-4d65-9146-7af562cab61d None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:24.984987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6bf8b8-7323-4d65-9146-7af562cab61d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:24.986611 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6bf8b8-7323-4d65-9146-7af562cab61d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:25.088746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 15/33] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Oct 6 12:07:24 2022] POST /identity/v3/roles => generated 225 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:07:25.088746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a127c5-6d42-4e9c-8730-b3d11cacd9ea None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:25.134401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:25.153043 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 19/34] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Oct 6 12:07:24 2022] POST /identity/v3/roles => generated 219 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:07:25.162456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6bf8b8-7323-4d65-9146-7af562cab61d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:25.289793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a127c5-6d42-4e9c-8730-b3d11cacd9ea None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:25.309235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:25.333621 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df46fb5-ecce-42dd-8b86-350d87751575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:25.334537 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df46fb5-ecce-42dd-8b86-350d87751575 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:25.334963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df46fb5-ecce-42dd-8b86-350d87751575 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:25.335377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df46fb5-ecce-42dd-8b86-350d87751575 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:25.336463 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df46fb5-ecce-42dd-8b86-350d87751575 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:25.336942 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df46fb5-ecce-42dd-8b86-350d87751575 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:25.339823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df46fb5-ecce-42dd-8b86-350d87751575 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:25.356973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 16/35] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:25.364630 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df46fb5-ecce-42dd-8b86-350d87751575 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:25.412715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6bf8b8-7323-4d65-9146-7af562cab61d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:25.471025 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884d006-1c9c-4092-8685-e54d1aa9db5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:25.471722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884d006-1c9c-4092-8685-e54d1aa9db5f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:25.472045 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884d006-1c9c-4092-8685-e54d1aa9db5f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:25.472296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884d006-1c9c-4092-8685-e54d1aa9db5f None admin] PATH_INFO: `/v3/users/50b3556af8274b8a9a59fe39e12271c7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:25.473633 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884d006-1c9c-4092-8685-e54d1aa9db5f None admin] RBAC: Authorizing `identity:get_user(user_id=50b3556af8274b8a9a59fe39e12271c7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:25.475418 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884d006-1c9c-4092-8685-e54d1aa9db5f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:25.477402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 20/36] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:25 2022] GET /identity/v3/users/50b3556af8274b8a9a59fe39e12271c7 => generated 247 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:25.483217 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884d006-1c9c-4092-8685-e54d1aa9db5f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:25.496596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:25.528305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df46fb5-ecce-42dd-8b86-350d87751575 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:25.575655 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bb99d-704b-4308-84ae-ec20386e43f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:25.576286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bb99d-704b-4308-84ae-ec20386e43f7 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:25.576564 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bb99d-704b-4308-84ae-ec20386e43f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:25.576842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bb99d-704b-4308-84ae-ec20386e43f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:25.577769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bb99d-704b-4308-84ae-ec20386e43f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:25.579229 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bb99d-704b-4308-84ae-ec20386e43f7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:25.619632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 17/37] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Oct 6 12:07:25 2022] POST /identity/v3/projects => generated 301 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:07:25.619823 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884d006-1c9c-4092-8685-e54d1aa9db5f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:25.619902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bb358-65c5-4632-921f-b813e436be04 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:25.619902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bb358-65c5-4632-921f-b813e436be04 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:25.619902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bb358-65c5-4632-921f-b813e436be04 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:25.619902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 18/38] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:25.625345 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9fa64-a9a9-45dc-8dc5-3e0a597bcdf4 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:25.625640 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9fa64-a9a9-45dc-8dc5-3e0a597bcdf4 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:25.625919 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9fa64-a9a9-45dc-8dc5-3e0a597bcdf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:25.647618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-2cb9fa64-a9a9-45dc-8dc5-3e0a597bcdf4 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:25.649214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2cb9fa64-a9a9-45dc-8dc5-3e0a597bcdf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:25.652034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:25.666062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d8a2f-46c5-4b36-9001-792230341638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:25.666940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d8a2f-46c5-4b36-9001-792230341638 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:25.668019 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d8a2f-46c5-4b36-9001-792230341638 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:25.668430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d8a2f-46c5-4b36-9001-792230341638 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:25.669946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d8a2f-46c5-4b36-9001-792230341638 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:25.671699 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d8a2f-46c5-4b36-9001-792230341638 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:25.674113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 21/39] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:07:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:25.679835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d8a2f-46c5-4b36-9001-792230341638 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:25.741741 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 19/40] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:07:25.909608 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d8a2f-46c5-4b36-9001-792230341638 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:25.945694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cff5d-4ead-44ad-bc3a-096ab49b841b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:25.946368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cff5d-4ead-44ad-bc3a-096ab49b841b None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:25.946654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cff5d-4ead-44ad-bc3a-096ab49b841b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:25.946935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cff5d-4ead-44ad-bc3a-096ab49b841b None admin] PATH_INFO: `/v3/domains/default/users/50b3556af8274b8a9a59fe39e12271c7/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:25.949516 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0cff5d-4ead-44ad-bc3a-096ab49b841b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=50b3556af8274b8a9a59fe39e12271c7, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:25.950979 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0cff5d-4ead-44ad-bc3a-096ab49b841b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:25.959833 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefba8b-12be-4d9f-8564-091bd80dd6b7 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:25.960122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefba8b-12be-4d9f-8564-091bd80dd6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:25.960404 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefba8b-12be-4d9f-8564-091bd80dd6b7 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:25.961798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 20/41] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:07:25.967004 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c1643-516d-4da5-aac3-2dc53cef6f2d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:25.967291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c1643-516d-4da5-aac3-2dc53cef6f2d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:25.967593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c1643-516d-4da5-aac3-2dc53cef6f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:25.970219 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 22/42] 10.222.0.33 () {58 vars in 1361 bytes} [Thu Oct 6 12:07:25 2022] PUT /identity/v3/domains/default/users/50b3556af8274b8a9a59fe39e12271c7/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:07:25.997674 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-808c1643-516d-4da5-aac3-2dc53cef6f2d None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:25.998913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-808c1643-516d-4da5-aac3-2dc53cef6f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:26.013004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cff5d-4ead-44ad-bc3a-096ab49b841b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:26.066817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:26.075008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:26.084576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 21/43] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:26.128888 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cff5d-4ead-44ad-bc3a-096ab49b841b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:26.168381 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0527a-c080-4e02-8353-7bc2b19f8641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:26.169234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0527a-c080-4e02-8353-7bc2b19f8641 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:26.169518 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0527a-c080-4e02-8353-7bc2b19f8641 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:26.169796 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0527a-c080-4e02-8353-7bc2b19f8641 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:26.170468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0527a-c080-4e02-8353-7bc2b19f8641 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:26.171909 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0527a-c080-4e02-8353-7bc2b19f8641 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:26.188747 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 23/44] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Oct 6 12:07:26 2022] POST /identity/v3/roles => generated 223 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:07:26.351417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c1643-516d-4da5-aac3-2dc53cef6f2d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:26.453509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c1643-516d-4da5-aac3-2dc53cef6f2d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:26.486117 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb7b17-9f1c-4f03-9b3d-d5de7e85a29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:26.486749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb7b17-9f1c-4f03-9b3d-d5de7e85a29d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:26.487032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb7b17-9f1c-4f03-9b3d-d5de7e85a29d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:26.487309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb7b17-9f1c-4f03-9b3d-d5de7e85a29d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:26.488438 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb7b17-9f1c-4f03-9b3d-d5de7e85a29d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:26.489886 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb7b17-9f1c-4f03-9b3d-d5de7e85a29d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:26.491446 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 22/45] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:07:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:26.497449 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0527a-c080-4e02-8353-7bc2b19f8641 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:26.595778 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0527a-c080-4e02-8353-7bc2b19f8641 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:26.609030 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:26.630964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbaa35-bd31-4d57-b63b-091137c3e313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:26.631605 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbaa35-bd31-4d57-b63b-091137c3e313 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:26.631892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbaa35-bd31-4d57-b63b-091137c3e313 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:26.632168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbaa35-bd31-4d57-b63b-091137c3e313 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:26.632812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbaa35-bd31-4d57-b63b-091137c3e313 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:26.634268 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbaa35-bd31-4d57-b63b-091137c3e313 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:26.663107 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 24/46] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Oct 6 12:07:26 2022] POST /identity/v3/projects => generated 312 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:07:28.115425 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32272fc8-0b47-4c90-9f66-7bddd48c9b3f None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:28.115991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32272fc8-0b47-4c90-9f66-7bddd48c9b3f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:28.115991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32272fc8-0b47-4c90-9f66-7bddd48c9b3f None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:28.117338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 23/47] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:28.123296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105436a-44a4-4cb8-b194-8c6ff1aaf142 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:28.123746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105436a-44a4-4cb8-b194-8c6ff1aaf142 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:28.123890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105436a-44a4-4cb8-b194-8c6ff1aaf142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:28.143236 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9105436a-44a4-4cb8-b194-8c6ff1aaf142 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:28.144773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9105436a-44a4-4cb8-b194-8c6ff1aaf142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:28.210736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 25/48] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:28.480783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32272fc8-0b47-4c90-9f66-7bddd48c9b3f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:28.539480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:28.581937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32272fc8-0b47-4c90-9f66-7bddd48c9b3f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:28.615263 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e72d50-cb20-4379-8858-829f1a9ab1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:28.615925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e72d50-cb20-4379-8858-829f1a9ab1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:28.616209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e72d50-cb20-4379-8858-829f1a9ab1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:28.616485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e72d50-cb20-4379-8858-829f1a9ab1d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:28.628208 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e72d50-cb20-4379-8858-829f1a9ab1d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:28.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-83e72d50-cb20-4379-8858-829f1a9ab1d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:07:28.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:28.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:28.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:28.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:28.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:28.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:28.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:28.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:28.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:28.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:28.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:28.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:28.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:28.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:28.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:28.635388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:28.635388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:28.635388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:28.635388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:28.635388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:28.635388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:28.635388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:28.635388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:28.635388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:28.635388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:28.635388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:28.635388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:28.635388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:28.635388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:28.635388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:28.635388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:28.636280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:28.636280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:28.636280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:28.636280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:28.636280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:28.636280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:28.636280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:28.636280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:28.636280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:28.636280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:28.636280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:07:28.636280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:07:28.636280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 24/49] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:07:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:28.641723 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105436a-44a4-4cb8-b194-8c6ff1aaf142 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:28.737000 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:28.744698 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105436a-44a4-4cb8-b194-8c6ff1aaf142 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:28.778428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2a304-d72f-4717-9cdf-0bb507dead03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:28.779043 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2a304-d72f-4717-9cdf-0bb507dead03 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:28.779327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2a304-d72f-4717-9cdf-0bb507dead03 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:28.779639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2a304-d72f-4717-9cdf-0bb507dead03 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:28.780657 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2a304-d72f-4717-9cdf-0bb507dead03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:28.780965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2a304-d72f-4717-9cdf-0bb507dead03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:28.782355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2a304-d72f-4717-9cdf-0bb507dead03 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:28.789463 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdf70e-d219-4a4f-9d46-441e6e9d2327 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:28.789754 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdf70e-d219-4a4f-9d46-441e6e9d2327 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:28.790032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdf70e-d219-4a4f-9d46-441e6e9d2327 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:28.790402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 26/50] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:07:28.791351 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 25/51] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:07:28.796430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2a304-d72f-4717-9cdf-0bb507dead03 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:28.796776 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f42674-bf24-45da-8e8c-af9794ec73c3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:28.797064 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f42674-bf24-45da-8e8c-af9794ec73c3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:28.797347 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f42674-bf24-45da-8e8c-af9794ec73c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:28.814908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-87f42674-bf24-45da-8e8c-af9794ec73c3 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:28.816116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-87f42674-bf24-45da-8e8c-af9794ec73c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:28.881418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 26/52] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:07:28.894831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2a304-d72f-4717-9cdf-0bb507dead03 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:28.928261 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848314df-4ade-4bd6-8335-01fb44ef35ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:28.929058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848314df-4ade-4bd6-8335-01fb44ef35ea None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:28.929429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848314df-4ade-4bd6-8335-01fb44ef35ea None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:28.929713 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848314df-4ade-4bd6-8335-01fb44ef35ea None admin] PATH_INFO: `/v3/users/50b3556af8274b8a9a59fe39e12271c7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:28.931099 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848314df-4ade-4bd6-8335-01fb44ef35ea None admin] RBAC: Authorizing `identity:get_user(user_id=50b3556af8274b8a9a59fe39e12271c7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:28.933088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848314df-4ade-4bd6-8335-01fb44ef35ea None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:28.939355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 27/53] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:28 2022] GET /identity/v3/users/50b3556af8274b8a9a59fe39e12271c7 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:28.940672 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f42674-bf24-45da-8e8c-af9794ec73c3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:28.990126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:29.041303 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f42674-bf24-45da-8e8c-af9794ec73c3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:29.154088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848314df-4ade-4bd6-8335-01fb44ef35ea None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:29.178145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc54a2-0202-4282-ba57-e6cbafd20855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:29.178781 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc54a2-0202-4282-ba57-e6cbafd20855 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:29.179061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc54a2-0202-4282-ba57-e6cbafd20855 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:29.179336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc54a2-0202-4282-ba57-e6cbafd20855 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:29.179717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:29.180476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc54a2-0202-4282-ba57-e6cbafd20855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:29.182044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc54a2-0202-4282-ba57-e6cbafd20855 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:29.184275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 27/54] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:07:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:29.189879 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc54a2-0202-4282-ba57-e6cbafd20855 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:29.255596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848314df-4ade-4bd6-8335-01fb44ef35ea None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:29.289146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7792ac-6e28-43d8-b38d-1bdc0f211cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:29.289773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7792ac-6e28-43d8-b38d-1bdc0f211cec None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:29.290052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7792ac-6e28-43d8-b38d-1bdc0f211cec None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:29.290322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7792ac-6e28-43d8-b38d-1bdc0f211cec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:29.291437 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7792ac-6e28-43d8-b38d-1bdc0f211cec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:29.293460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7792ac-6e28-43d8-b38d-1bdc0f211cec None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:29.294616 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc54a2-0202-4282-ba57-e6cbafd20855 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:29.295016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 28/55] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:07:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:07:29.300410 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7792ac-6e28-43d8-b38d-1bdc0f211cec None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:29.310485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:29.329223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a556e47-8443-46bf-bed4-6bc616e49a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:29.329824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a556e47-8443-46bf-bed4-6bc616e49a65 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:29.330105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a556e47-8443-46bf-bed4-6bc616e49a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:29.330373 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a556e47-8443-46bf-bed4-6bc616e49a65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:29.331087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a556e47-8443-46bf-bed4-6bc616e49a65 None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.domain.id=default, user.id=50b3556af8274b8a9a59fe39e12271c7) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:29.331393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a556e47-8443-46bf-bed4-6bc616e49a65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:29.332837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a556e47-8443-46bf-bed4-6bc616e49a65 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:29.341756 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 28/56] 10.222.0.33 () {56 vars in 1322 bytes} [Thu Oct 6 12:07:29 2022] GET /identity/v3/role_assignments?user.id=50b3556af8274b8a9a59fe39e12271c7&scope.domain.id=default&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:07:29.394314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:29.404757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7792ac-6e28-43d8-b38d-1bdc0f211cec None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:29.440907 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32ffd3-3679-4fe4-96b1-8f0e0ee4def8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:29.441534 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32ffd3-3679-4fe4-96b1-8f0e0ee4def8 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:29.441816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32ffd3-3679-4fe4-96b1-8f0e0ee4def8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:29.442092 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32ffd3-3679-4fe4-96b1-8f0e0ee4def8 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:29.442748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32ffd3-3679-4fe4-96b1-8f0e0ee4def8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:29.444279 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32ffd3-3679-4fe4-96b1-8f0e0ee4def8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:29.473407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 29/57] 10.222.0.33 () {60 vars in 1140 bytes} [Thu Oct 6 12:07:29 2022] POST /identity/v3/projects => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:07:31.609253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59be28-345c-46f4-b48f-9a5bd9e2c5a1 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:31.609748 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59be28-345c-46f4-b48f-9a5bd9e2c5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:31.609829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59be28-345c-46f4-b48f-9a5bd9e2c5a1 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:31.611151 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 29/58] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:31.616214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39a6b1-4a02-4b4b-82ff-84ce2062f141 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:31.616596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39a6b1-4a02-4b4b-82ff-84ce2062f141 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:31.616762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39a6b1-4a02-4b4b-82ff-84ce2062f141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:31.634249 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-eb39a6b1-4a02-4b4b-82ff-84ce2062f141 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:31.635426 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-eb39a6b1-4a02-4b4b-82ff-84ce2062f141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:31.700893 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 30/59] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:31.967688 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59be28-345c-46f4-b48f-9a5bd9e2c5a1 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:32.066175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59be28-345c-46f4-b48f-9a5bd9e2c5a1 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:32.098749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc520f-a5bc-48b3-a343-70aefcb3b881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:32.099365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc520f-a5bc-48b3-a343-70aefcb3b881 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:32.099684 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc520f-a5bc-48b3-a343-70aefcb3b881 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:32.099965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc520f-a5bc-48b3-a343-70aefcb3b881 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:32.101085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc520f-a5bc-48b3-a343-70aefcb3b881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:32.102535 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc520f-a5bc-48b3-a343-70aefcb3b881 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:32.104110 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 30/60] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:07:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:32.110198 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39a6b1-4a02-4b4b-82ff-84ce2062f141 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:32.180709 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:32.206144 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39a6b1-4a02-4b4b-82ff-84ce2062f141 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:32.238075 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf1ce6-22af-4469-8895-cc8c72d0ab01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:32.238630 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf1ce6-22af-4469-8895-cc8c72d0ab01 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:32.238910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf1ce6-22af-4469-8895-cc8c72d0ab01 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:32.239184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf1ce6-22af-4469-8895-cc8c72d0ab01 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:32.239869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf1ce6-22af-4469-8895-cc8c72d0ab01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:32.241304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf1ce6-22af-4469-8895-cc8c72d0ab01 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:32.267291 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 31/61] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:07:32 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:07:35.723175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d80924-ecd4-4c4f-8526-38085d2ec931 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:35.723719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5967c0c-9ee4-4733-8b5c-4eedd5c225dd None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:35.723719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5967c0c-9ee4-4733-8b5c-4eedd5c225dd None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:35.724143 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d80924-ecd4-4c4f-8526-38085d2ec931 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:35.724223 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5967c0c-9ee4-4733-8b5c-4eedd5c225dd None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:35.724291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d80924-ecd4-4c4f-8526-38085d2ec931 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:35.725392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 32/63] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:35.725496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 31/63] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:35.730538 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022bb27-8b16-465b-bbbf-c7dc6719da57 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:35.731086 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022bb27-8b16-465b-bbbf-c7dc6719da57 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:35.731605 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022bb27-8b16-465b-bbbf-c7dc6719da57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:35.736743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510d2d4-db66-471e-b242-d44badb386f1 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:35.737269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510d2d4-db66-471e-b242-d44badb386f1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:35.737745 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510d2d4-db66-471e-b242-d44badb386f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:35.758934 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f510d2d4-db66-471e-b242-d44badb386f1 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:35.759489 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-1022bb27-8b16-465b-bbbf-c7dc6719da57 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:35.760727 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1022bb27-8b16-465b-bbbf-c7dc6719da57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:35.761431 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f510d2d4-db66-471e-b242-d44badb386f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:35.808617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:35.834554 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 32/64] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 06 12:07:35.841750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742dc5f-3e64-41b4-9ef2-51dfcd563106 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:35.841750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742dc5f-3e64-41b4-9ef2-51dfcd563106 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:35.841750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742dc5f-3e64-41b4-9ef2-51dfcd563106 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:35.843648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 33/65] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:35.848459 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8183152-8fe6-442a-81ee-631a94bd6383 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:35.848752 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8183152-8fe6-442a-81ee-631a94bd6383 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:35.849038 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8183152-8fe6-442a-81ee-631a94bd6383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:35.852584 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 33/66] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:35.888536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c8183152-8fe6-442a-81ee-631a94bd6383 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:35.889845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c8183152-8fe6-442a-81ee-631a94bd6383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:35.982202 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 34/67] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:36.133227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dbe1d-9002-462a-a261-56859df4b056 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:36.133532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dbe1d-9002-462a-a261-56859df4b056 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:36.133776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dbe1d-9002-462a-a261-56859df4b056 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:36.135136 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 34/68] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:36.140493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74824509-e13f-4f79-a80c-24734cfba4dd None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:36.141018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74824509-e13f-4f79-a80c-24734cfba4dd None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:36.141509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74824509-e13f-4f79-a80c-24734cfba4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:36.157391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141d0aa-6e5e-43d3-8e74-8ece5af4fe5b None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:36.157682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141d0aa-6e5e-43d3-8e74-8ece5af4fe5b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:36.157963 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141d0aa-6e5e-43d3-8e74-8ece5af4fe5b None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:36.159919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 35/69] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:36.163798 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141d0aa-6e5e-43d3-8e74-8ece5af4fe5b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:36.167469 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-74824509-e13f-4f79-a80c-24734cfba4dd None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:36.168938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-74824509-e13f-4f79-a80c-24734cfba4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:36.239676 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:36.242725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 35/70] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:36.244226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dde69-73bd-40e5-a330-bc365aa0f4c4 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:36.244780 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dde69-73bd-40e5-a330-bc365aa0f4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:36.245685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dde69-73bd-40e5-a330-bc365aa0f4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:36.265770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-0e2dde69-73bd-40e5-a330-bc365aa0f4c4 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:36.266948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0e2dde69-73bd-40e5-a330-bc365aa0f4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:36.301947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:36.305661 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141d0aa-6e5e-43d3-8e74-8ece5af4fe5b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:36.340204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d93a08-2fd7-4b69-8afc-04c231fd63b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:36.340840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d93a08-2fd7-4b69-8afc-04c231fd63b3 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:36.341170 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d93a08-2fd7-4b69-8afc-04c231fd63b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:36.341445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d93a08-2fd7-4b69-8afc-04c231fd63b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:36.350531 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 36/71] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:36.351632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dde69-73bd-40e5-a330-bc365aa0f4c4 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:36.352266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d93a08-2fd7-4b69-8afc-04c231fd63b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:36.357749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-36d93a08-2fd7-4b69-8afc-04c231fd63b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:07:36.357749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:36.357749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:36.357749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:36.357749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:36.357749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:36.357749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:36.357749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:36.357749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:36.357749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:36.357749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:36.357749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:36.357749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:36.357749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:36.357749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:36.357749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:36.358613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:36.358613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:36.358613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:36.358613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:36.358613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:36.358613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:36.358613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:36.358613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:36.358613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:36.358613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:36.358613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:36.358613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:36.358613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:36.358613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:36.358613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:36.358613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:36.359387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:36.359387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:36.359387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:36.359387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:36.359387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:36.359387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:36.359387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:36.359387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:36.359387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:36.359387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:36.359387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:07:36.359387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:07:36.363630 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 36/72] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:07:36.364464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d93a08-2fd7-4b69-8afc-04c231fd63b3 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:36.477885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dde69-73bd-40e5-a330-bc365aa0f4c4 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:36.483254 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d93a08-2fd7-4b69-8afc-04c231fd63b3 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:36.507353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:36.520916 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d0963-43b4-4840-b103-7bc5e5795437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:36.521836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d0963-43b4-4840-b103-7bc5e5795437 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:36.522165 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408691a6-f5c0-436e-aba2-8b69e0581c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:36.522561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d0963-43b4-4840-b103-7bc5e5795437 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:36.522890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408691a6-f5c0-436e-aba2-8b69e0581c96 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:36.523173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408691a6-f5c0-436e-aba2-8b69e0581c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:36.523436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d0963-43b4-4840-b103-7bc5e5795437 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:36.523557 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408691a6-f5c0-436e-aba2-8b69e0581c96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:36.534898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408691a6-f5c0-436e-aba2-8b69e0581c96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:36.535832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d0963-43b4-4840-b103-7bc5e5795437 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:36.539724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-408691a6-f5c0-436e-aba2-8b69e0581c96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:07:36.539724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:36.539724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:36.539724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:36.539724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:36.539724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:36.539724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:36.539724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:36.539724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:36.539724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:36.539724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:36.539724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:36.539724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:36.539724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:36.539724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:36.539724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:36.540569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:36.540569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:36.540569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:36.540569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:36.540569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:36.540569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:36.540569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:36.540569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:36.540569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:36.540569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:36.540569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:36.540569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:36.540569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:36.540569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:36.540569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:36.540569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:36.541422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:36.541422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:36.541422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:36.541422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:36.541422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:36.541422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:36.541422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:36.541422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:36.541422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:36.541422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:36.541422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:07:36.541422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:07:36.541422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 37/73] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:07:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:36.542634 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408691a6-f5c0-436e-aba2-8b69e0581c96 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:36.544223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-625d0963-43b4-4840-b103-7bc5e5795437 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:07:36.544223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:36.544223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:36.544223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:36.544223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:36.544223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:36.544223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:36.544223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:36.544223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:36.544223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:36.544223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:36.544223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:36.544223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:36.544223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:36.544223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:36.544223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:36.545060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:36.545060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:36.545060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:36.545060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:36.545060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:36.545060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:36.545060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:36.545060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:36.545060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:36.545060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:36.545060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:36.545060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:36.545060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:36.545060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:36.545060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:36.545060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:36.545850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:36.545850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:36.545850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:36.545850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:36.545850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:36.545850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:36.545850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:36.545850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:36.545850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:36.545850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:36.545850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:07:36.545850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:07:36.547802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 37/74] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:07:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:36.548772 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d0963-43b4-4840-b103-7bc5e5795437 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:36.579352 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:36.647686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408691a6-f5c0-436e-aba2-8b69e0581c96 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:36.658085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d0963-43b4-4840-b103-7bc5e5795437 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:36.681535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a0d3b-ae4b-48b9-b6d8-fa2c4f6f6dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:36.682150 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a0d3b-ae4b-48b9-b6d8-fa2c4f6f6dec None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:36.682439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a0d3b-ae4b-48b9-b6d8-fa2c4f6f6dec None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:36.682744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a0d3b-ae4b-48b9-b6d8-fa2c4f6f6dec None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:36.683411 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a0d3b-ae4b-48b9-b6d8-fa2c4f6f6dec None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:36.683753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a0d3b-ae4b-48b9-b6d8-fa2c4f6f6dec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:36.685160 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a0d3b-ae4b-48b9-b6d8-fa2c4f6f6dec None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:36.692171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be4be2-d192-48b0-b301-1bd10dd3fa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:36.692798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be4be2-d192-48b0-b301-1bd10dd3fa1d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:36.693067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 38/75] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:07:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:36.693150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be4be2-d192-48b0-b301-1bd10dd3fa1d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:36.693937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a0d3b-ae4b-48b9-b6d8-fa2c4f6f6dec None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:36.695874 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be4be2-d192-48b0-b301-1bd10dd3fa1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:36.698035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be4be2-d192-48b0-b301-1bd10dd3fa1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:36.700071 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be4be2-d192-48b0-b301-1bd10dd3fa1d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:36.701700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 38/76] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:07:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:36.702459 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be4be2-d192-48b0-b301-1bd10dd3fa1d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:36.737134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:36.794691 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a0d3b-ae4b-48b9-b6d8-fa2c4f6f6dec None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:36.795871 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:36.802739 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be4be2-d192-48b0-b301-1bd10dd3fa1d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:36.826929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a8b7d-d31b-492b-b1b3-d109cddeb899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:36.827571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a8b7d-d31b-492b-b1b3-d109cddeb899 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:36.827857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a8b7d-d31b-492b-b1b3-d109cddeb899 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:36.828134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a8b7d-d31b-492b-b1b3-d109cddeb899 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:36.828797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a8b7d-d31b-492b-b1b3-d109cddeb899 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:36.829091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a8b7d-d31b-492b-b1b3-d109cddeb899 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:36.830458 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a8b7d-d31b-492b-b1b3-d109cddeb899 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:36.836963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1355fd7-ee20-4141-9133-6b3254e90dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:36.837559 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1355fd7-ee20-4141-9133-6b3254e90dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:36.837845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1355fd7-ee20-4141-9133-6b3254e90dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:36.838120 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1355fd7-ee20-4141-9133-6b3254e90dd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:36.838591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 39/77] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:36.838923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1355fd7-ee20-4141-9133-6b3254e90dd6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:36.839238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1355fd7-ee20-4141-9133-6b3254e90dd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:36.839447 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a8b7d-d31b-492b-b1b3-d109cddeb899 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:36.840705 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1355fd7-ee20-4141-9133-6b3254e90dd6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:36.852433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 39/78] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:07:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:36.853240 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1355fd7-ee20-4141-9133-6b3254e90dd6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:36.942164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a8b7d-d31b-492b-b1b3-d109cddeb899 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:36.945180 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:36.950914 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1355fd7-ee20-4141-9133-6b3254e90dd6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:36.975828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3e65a-284d-41a0-ada6-f83759bbca9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:36.976477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3e65a-284d-41a0-ada6-f83759bbca9c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:36.976748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3e65a-284d-41a0-ada6-f83759bbca9c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:36.977028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3e65a-284d-41a0-ada6-f83759bbca9c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:36.985245 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9a498-d821-4c68-999f-beebd426eb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:36.985863 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9a498-d821-4c68-999f-beebd426eb70 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:36.986144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9a498-d821-4c68-999f-beebd426eb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:36.986419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9a498-d821-4c68-999f-beebd426eb70 None admin] PATH_INFO: `/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:36.986981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3e65a-284d-41a0-ada6-f83759bbca9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:36.992323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-c5a3e65a-284d-41a0-ada6-f83759bbca9c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:07:36.992323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:36.992323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:36.992323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:36.992323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:36.992323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:36.992323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:36.992323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:36.992323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:36.992323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:36.992323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:36.992323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:36.992323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:36.992323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:36.992323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:36.992323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:36.993206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:36.993206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:36.993206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:36.993206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:36.993206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:36.993206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:36.993206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:36.993206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:36.993206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:36.993206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:36.993206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:36.993206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:36.993206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:36.993206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:36.993206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:36.993206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:36.994190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:36.994190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:36.994190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:36.994190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:36.994190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:36.994190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:36.994190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:36.994190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:36.994190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:36.994190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:36.994190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:07:36.994190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:07:36.994190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 40/79] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:36.994190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3e65a-284d-41a0-ada6-f83759bbca9c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:37.006377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9a498-d821-4c68-999f-beebd426eb70 None admin] RBAC: Authorizing `identity:get_user(user_id=02b6e37825904c729c9cf6fd8e1f7dbd)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:37.007870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9a498-d821-4c68-999f-beebd426eb70 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:37.009590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 40/80] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:36 2022] GET /identity/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:37.010337 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9a498-d821-4c68-999f-beebd426eb70 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:37.028093 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:37.096792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3e65a-284d-41a0-ada6-f83759bbca9c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:37.111246 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9a498-d821-4c68-999f-beebd426eb70 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:37.128853 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3606b9-fadc-4733-84c5-a1351f724378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:37.129453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3606b9-fadc-4733-84c5-a1351f724378 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:37.129733 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3606b9-fadc-4733-84c5-a1351f724378 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:37.130004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3606b9-fadc-4733-84c5-a1351f724378 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:37.130650 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3606b9-fadc-4733-84c5-a1351f724378 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:37.132132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3606b9-fadc-4733-84c5-a1351f724378 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:37.144903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647134d-9647-415a-9958-597856ff118c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:37.145558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647134d-9647-415a-9958-597856ff118c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:37.145850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647134d-9647-415a-9958-597856ff118c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:37.146135 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647134d-9647-415a-9958-597856ff118c None admin] PATH_INFO: `/v3/users/50b3556af8274b8a9a59fe39e12271c7` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:37.147337 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647134d-9647-415a-9958-597856ff118c None admin] RBAC: Authorizing `identity:get_user(user_id=50b3556af8274b8a9a59fe39e12271c7)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:37.148811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647134d-9647-415a-9958-597856ff118c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:37.150398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 41/81] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:37 2022] GET /identity/v3/users/50b3556af8274b8a9a59fe39e12271c7 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:37.151165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647134d-9647-415a-9958-597856ff118c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:37.159556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 41/82] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:07:36 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:07:37.159556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3606b9-fadc-4733-84c5-a1351f724378 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:37.183265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:37.267225 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:37.282520 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3606b9-fadc-4733-84c5-a1351f724378 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:37.288603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647134d-9647-415a-9958-597856ff118c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:37.322972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb671fc8-f096-4f3b-bae0-fab1971bc191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:37.322972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb671fc8-f096-4f3b-bae0-fab1971bc191 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:37.322972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb671fc8-f096-4f3b-bae0-fab1971bc191 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:37.322972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb671fc8-f096-4f3b-bae0-fab1971bc191 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:37.323903 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb671fc8-f096-4f3b-bae0-fab1971bc191 None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:37.323903 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb671fc8-f096-4f3b-bae0-fab1971bc191 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:37.324461 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb671fc8-f096-4f3b-bae0-fab1971bc191 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:37.329875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9a86c-f56c-43ba-9cd4-71042848d87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:37.329875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9a86c-f56c-43ba-9cd4-71042848d87d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:37.329875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9a86c-f56c-43ba-9cd4-71042848d87d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:37.330132 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9a86c-f56c-43ba-9cd4-71042848d87d None admin] PATH_INFO: `/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:37.331840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9a86c-f56c-43ba-9cd4-71042848d87d None admin] RBAC: Authorizing `identity:get_user(user_id=02b6e37825904c729c9cf6fd8e1f7dbd)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:37.332692 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9a86c-f56c-43ba-9cd4-71042848d87d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:37.334150 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 42/83] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:07:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:37.334253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 42/84] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:37 2022] GET /identity/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:37.339823 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb671fc8-f096-4f3b-bae0-fab1971bc191 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:37.339947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9a86c-f56c-43ba-9cd4-71042848d87d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:37.439954 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:37.464990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb671fc8-f096-4f3b-bae0-fab1971bc191 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:37.465171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9a86c-f56c-43ba-9cd4-71042848d87d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:37.508137 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7264d-8179-45ae-968c-32e56a0f7d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:37.508767 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7264d-8179-45ae-968c-32e56a0f7d5f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:37.509301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7264d-8179-45ae-968c-32e56a0f7d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:37.509604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7264d-8179-45ae-968c-32e56a0f7d5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:37.516245 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9770914-8344-46ec-9541-da9559a6629e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:37.516873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9770914-8344-46ec-9541-da9559a6629e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:37.517164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9770914-8344-46ec-9541-da9559a6629e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:37.517445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9770914-8344-46ec-9541-da9559a6629e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:37.536090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:37.586260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7264d-8179-45ae-968c-32e56a0f7d5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:37.587802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7264d-8179-45ae-968c-32e56a0f7d5f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:37.609121 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9770914-8344-46ec-9541-da9559a6629e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:37.611656 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9770914-8344-46ec-9541-da9559a6629e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:37.708843 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 43/85] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:07:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:07:37.709419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7264d-8179-45ae-968c-32e56a0f7d5f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:37.723795 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:37.752457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 43/86] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:07:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:07:37.753963 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9770914-8344-46ec-9541-da9559a6629e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:37.771025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:37.825823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7264d-8179-45ae-968c-32e56a0f7d5f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:37.864009 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af2af6-e11d-4cf9-99a5-aef251b211c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:37.864649 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af2af6-e11d-4cf9-99a5-aef251b211c0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:37.865157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af2af6-e11d-4cf9-99a5-aef251b211c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:37.865270 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af2af6-e11d-4cf9-99a5-aef251b211c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:37.887689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9770914-8344-46ec-9541-da9559a6629e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:37.932290 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511cbfb-5bc8-4c3c-bd6b-4485f5a6c09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:37.934342 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511cbfb-5bc8-4c3c-bd6b-4485f5a6c09d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:37.934775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511cbfb-5bc8-4c3c-bd6b-4485f5a6c09d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:37.935191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511cbfb-5bc8-4c3c-bd6b-4485f5a6c09d None admin] PATH_INFO: `/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:37.937285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511cbfb-5bc8-4c3c-bd6b-4485f5a6c09d None admin] RBAC: Authorizing `identity:get_user(user_id=02b6e37825904c729c9cf6fd8e1f7dbd)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:37.938885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511cbfb-5bc8-4c3c-bd6b-4485f5a6c09d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:37.941481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 44/87] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:37 2022] GET /identity/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:37.942795 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af2af6-e11d-4cf9-99a5-aef251b211c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:37.943017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511cbfb-5bc8-4c3c-bd6b-4485f5a6c09d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:37.944289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af2af6-e11d-4cf9-99a5-aef251b211c0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:38.005884 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:38.010020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:38.067155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 44/88] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:07:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:07:38.067155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af2af6-e11d-4cf9-99a5-aef251b211c0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:38.085581 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511cbfb-5bc8-4c3c-bd6b-4485f5a6c09d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:38.131214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549eacfa-0960-452f-8c07-f44fcd09aff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:38.132696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549eacfa-0960-452f-8c07-f44fcd09aff3 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:38.133253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549eacfa-0960-452f-8c07-f44fcd09aff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:38.133673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549eacfa-0960-452f-8c07-f44fcd09aff3 None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:38.136145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549eacfa-0960-452f-8c07-f44fcd09aff3 None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:38.138029 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549eacfa-0960-452f-8c07-f44fcd09aff3 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:38.140141 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 45/89] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:37 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:38.141056 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549eacfa-0960-452f-8c07-f44fcd09aff3 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:38.193499 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af2af6-e11d-4cf9-99a5-aef251b211c0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:38.254796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42746f72-e00b-4f4a-89f7-dc089aaeb0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:38.255970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42746f72-e00b-4f4a-89f7-dc089aaeb0ce None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:38.256417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42746f72-e00b-4f4a-89f7-dc089aaeb0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:38.256837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42746f72-e00b-4f4a-89f7-dc089aaeb0ce None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:38.257450 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549eacfa-0960-452f-8c07-f44fcd09aff3 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:38.258271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42746f72-e00b-4f4a-89f7-dc089aaeb0ce None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:38.261578 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42746f72-e00b-4f4a-89f7-dc089aaeb0ce None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:38.263398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 45/90] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:38 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:38.264326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42746f72-e00b-4f4a-89f7-dc089aaeb0ce None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:38.281703 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:38.294183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:38.306234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab3ce0-f428-4604-828f-af7a4eccb478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:38.306901 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab3ce0-f428-4604-828f-af7a4eccb478 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:38.307187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab3ce0-f428-4604-828f-af7a4eccb478 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:38.307458 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab3ce0-f428-4604-828f-af7a4eccb478 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:38.388369 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42746f72-e00b-4f4a-89f7-dc089aaeb0ce None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:38.393577 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab3ce0-f428-4604-828f-af7a4eccb478 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:38.395053 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab3ce0-f428-4604-828f-af7a4eccb478 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:38.433169 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b77a5-5a3d-4dcc-a251-311293f08bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:38.434154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b77a5-5a3d-4dcc-a251-311293f08bfe None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:38.434633 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b77a5-5a3d-4dcc-a251-311293f08bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:38.435531 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b77a5-5a3d-4dcc-a251-311293f08bfe None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:38.437252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b77a5-5a3d-4dcc-a251-311293f08bfe None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:38.439570 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b77a5-5a3d-4dcc-a251-311293f08bfe None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:38.441602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 46/91] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:38 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:38.443013 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b77a5-5a3d-4dcc-a251-311293f08bfe None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:38.497970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 46/92] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:07:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:07:38.499288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab3ce0-f428-4604-828f-af7a4eccb478 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:38.517731 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:38.558865 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:38.569795 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b77a5-5a3d-4dcc-a251-311293f08bfe None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:38.606296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb74391-2d48-4dd1-a4c1-d5d3a3ad4ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:38.606948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb74391-2d48-4dd1-a4c1-d5d3a3ad4ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:38.607238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb74391-2d48-4dd1-a4c1-d5d3a3ad4ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:38.607531 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb74391-2d48-4dd1-a4c1-d5d3a3ad4ff3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:38.608285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb74391-2d48-4dd1-a4c1-d5d3a3ad4ff3 None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=b427802e13664086974c8d53f5bfbc88, user.id=50b3556af8274b8a9a59fe39e12271c7) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:38.608597 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb74391-2d48-4dd1-a4c1-d5d3a3ad4ff3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:38.610041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb74391-2d48-4dd1-a4c1-d5d3a3ad4ff3 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:38.615706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 47/93] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:07:38 2022] GET /identity/v3/role_assignments?user.id=50b3556af8274b8a9a59fe39e12271c7&scope.project.id=b427802e13664086974c8d53f5bfbc88&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:38.617055 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb74391-2d48-4dd1-a4c1-d5d3a3ad4ff3 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:38.622984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab3ce0-f428-4604-828f-af7a4eccb478 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:38.671871 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32b531-faa7-4bdd-bb72-aeb3f7400275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:38.672733 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32b531-faa7-4bdd-bb72-aeb3f7400275 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:38.673402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32b531-faa7-4bdd-bb72-aeb3f7400275 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:38.673687 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32b531-faa7-4bdd-bb72-aeb3f7400275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:38.674431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32b531-faa7-4bdd-bb72-aeb3f7400275 None admin] RBAC: Adding query filter params (role.id=bbea734b8a704d7ca8ec61bd45536cbe, scope.project.id=b427802e13664086974c8d53f5bfbc88, user.id=02b6e37825904c729c9cf6fd8e1f7dbd) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:38.674744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32b531-faa7-4bdd-bb72-aeb3f7400275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:38.680403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32b531-faa7-4bdd-bb72-aeb3f7400275 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:38.694337 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 47/94] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:07:38 2022] GET /identity/v3/role_assignments?user.id=02b6e37825904c729c9cf6fd8e1f7dbd&scope.project.id=b427802e13664086974c8d53f5bfbc88&role.id=bbea734b8a704d7ca8ec61bd45536cbe => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:38.696018 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32b531-faa7-4bdd-bb72-aeb3f7400275 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:38.818829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb74391-2d48-4dd1-a4c1-d5d3a3ad4ff3 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:38.895066 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfc17a-a9df-459d-b8d3-8611f61e9b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:38.895722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc17a-a9df-459d-b8d3-8611f61e9b2f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:38.895992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc17a-a9df-459d-b8d3-8611f61e9b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:38.896267 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc17a-a9df-459d-b8d3-8611f61e9b2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:38.897009 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfc17a-a9df-459d-b8d3-8611f61e9b2f None admin] RBAC: Adding query filter params (role.id=2c11a95c1ecf4988a95aad8d96ea56e9, scope.project.id=b427802e13664086974c8d53f5bfbc88, user.id=02b6e37825904c729c9cf6fd8e1f7dbd) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:38.897422 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfc17a-a9df-459d-b8d3-8611f61e9b2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:38.898907 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfc17a-a9df-459d-b8d3-8611f61e9b2f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:38.909692 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 48/95] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:07:38 2022] GET /identity/v3/role_assignments?user.id=02b6e37825904c729c9cf6fd8e1f7dbd&scope.project.id=b427802e13664086974c8d53f5bfbc88&role.id=2c11a95c1ecf4988a95aad8d96ea56e9 => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:38.915924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:38.935970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32b531-faa7-4bdd-bb72-aeb3f7400275 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:38.974994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb150850-6bf0-4c00-b204-02860896e1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:38.975732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb150850-6bf0-4c00-b204-02860896e1ad None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:38.976016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb150850-6bf0-4c00-b204-02860896e1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:38.976292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb150850-6bf0-4c00-b204-02860896e1ad None admin] PATH_INFO: `/v3/projects/f9c2c60f747b4390b69a797a1d5781f7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:38.977553 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb150850-6bf0-4c00-b204-02860896e1ad None admin] RBAC: Authorizing `identity:get_project(project_id=f9c2c60f747b4390b69a797a1d5781f7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:38.979030 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb150850-6bf0-4c00-b204-02860896e1ad None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:38.981498 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 48/96] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:38 2022] GET /identity/v3/projects/f9c2c60f747b4390b69a797a1d5781f7 => generated 312 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:38.987431 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfc17a-a9df-459d-b8d3-8611f61e9b2f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:39.014398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:39.133205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfc17a-a9df-459d-b8d3-8611f61e9b2f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:39.179686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b256957-3d6e-4815-bf96-a69106d53bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:39.180297 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b256957-3d6e-4815-bf96-a69106d53bce None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:39.180580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b256957-3d6e-4815-bf96-a69106d53bce None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:39.180857 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b256957-3d6e-4815-bf96-a69106d53bce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:39.181580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b256957-3d6e-4815-bf96-a69106d53bce None admin] RBAC: Adding query filter params (role.id=bbea734b8a704d7ca8ec61bd45536cbe, scope.project.id=f9c2c60f747b4390b69a797a1d5781f7, user.id=02b6e37825904c729c9cf6fd8e1f7dbd) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:39.181883 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b256957-3d6e-4815-bf96-a69106d53bce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:39.183955 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b256957-3d6e-4815-bf96-a69106d53bce None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:39.195614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 49/97] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:07:38 2022] GET /identity/v3/role_assignments?user.id=02b6e37825904c729c9cf6fd8e1f7dbd&scope.project.id=f9c2c60f747b4390b69a797a1d5781f7&role.id=bbea734b8a704d7ca8ec61bd45536cbe => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:39.781853 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34548ff-2d56-4b3a-9c54-b66a44d27078 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:39.782230 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34548ff-2d56-4b3a-9c54-b66a44d27078 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:39.782416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34548ff-2d56-4b3a-9c54-b66a44d27078 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:39.783765 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 49/98] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:39.790040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c719200-54e2-4df3-a2eb-29f66678f9b9 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:39.790323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c719200-54e2-4df3-a2eb-29f66678f9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:39.790593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c719200-54e2-4df3-a2eb-29f66678f9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:39.816935 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3c719200-54e2-4df3-a2eb-29f66678f9b9 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:39.818139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3c719200-54e2-4df3-a2eb-29f66678f9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:39.897923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 50/99] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:07:39.939060 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ac1bd-4ef3-443a-96dc-21105d6a33af None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:39.940020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ac1bd-4ef3-443a-96dc-21105d6a33af None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:39.940516 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ac1bd-4ef3-443a-96dc-21105d6a33af None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:39.942043 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 50/100] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:39.947954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baff63f-b853-4cc8-b1a1-b699637e9ee3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:39.948540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baff63f-b853-4cc8-b1a1-b699637e9ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:39.949007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baff63f-b853-4cc8-b1a1-b699637e9ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:39.968252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-1baff63f-b853-4cc8-b1a1-b699637e9ee3 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:39.969828 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1baff63f-b853-4cc8-b1a1-b699637e9ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:39.973200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:40.054333 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 51/101] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:40.359025 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ac1bd-4ef3-443a-96dc-21105d6a33af None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:40.498801 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ac1bd-4ef3-443a-96dc-21105d6a33af None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:40.542535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883c78c-a5ac-4e94-b41d-d7fe79f5a0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:40.543180 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883c78c-a5ac-4e94-b41d-d7fe79f5a0cb None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:40.543462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883c78c-a5ac-4e94-b41d-d7fe79f5a0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:40.543830 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883c78c-a5ac-4e94-b41d-d7fe79f5a0cb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:40.554367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2883c78c-a5ac-4e94-b41d-d7fe79f5a0cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:40.559892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-2883c78c-a5ac-4e94-b41d-d7fe79f5a0cb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:07:40.559892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:40.559892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:40.559892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:40.559892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:40.559892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:40.559892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:40.559892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:40.559892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:40.559892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:40.559892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:40.559892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:40.559892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:40.559892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:40.559892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:40.559892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:40.560736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:40.560736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:40.560736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:40.560736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:40.560736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:40.560736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:40.560736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:40.560736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:40.560736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:40.560736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:40.560736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:40.560736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:40.560736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:40.560736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:40.560736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:40.560736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:40.561523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:40.561523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:40.561523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:40.561523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:40.561523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:40.561523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:40.561523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:40.561523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:40.561523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:40.561523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:40.561523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:07:40.561523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:07:40.561523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 51/102] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:40.566867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baff63f-b853-4cc8-b1a1-b699637e9ee3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:40.632402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883c78c-a5ac-4e94-b41d-d7fe79f5a0cb None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:40.656466 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:40.678480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baff63f-b853-4cc8-b1a1-b699637e9ee3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:40.716607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ade84-2f2d-4b50-b1dd-79efea21f8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:40.717229 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ade84-2f2d-4b50-b1dd-79efea21f8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:40.717516 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ade84-2f2d-4b50-b1dd-79efea21f8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:40.717803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ade84-2f2d-4b50-b1dd-79efea21f8e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:40.718478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ade84-2f2d-4b50-b1dd-79efea21f8e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:40.718789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ade84-2f2d-4b50-b1dd-79efea21f8e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:40.720344 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ade84-2f2d-4b50-b1dd-79efea21f8e9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:40.729176 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 52/103] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:07:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:40.736172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ade84-2f2d-4b50-b1dd-79efea21f8e9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:40.738893 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:40.844678 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ade84-2f2d-4b50-b1dd-79efea21f8e9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:40.848912 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883c78c-a5ac-4e94-b41d-d7fe79f5a0cb None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:40.881091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8b677-c39a-40d4-9788-36ebbb799684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:40.881881 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8b677-c39a-40d4-9788-36ebbb799684 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:40.882174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8b677-c39a-40d4-9788-36ebbb799684 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:40.882512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8b677-c39a-40d4-9788-36ebbb799684 None admin] PATH_INFO: `/v3/users/1cbd4e62c7a0428ba026ac3c642204c9` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:40.897477 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8b677-c39a-40d4-9788-36ebbb799684 None admin] RBAC: Authorizing `identity:get_user(user_id=1cbd4e62c7a0428ba026ac3c642204c9)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:40.899273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8b677-c39a-40d4-9788-36ebbb799684 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:40.900977 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 53/104] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:40 2022] GET /identity/v3/users/1cbd4e62c7a0428ba026ac3c642204c9 => generated 289 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:40.908726 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8b677-c39a-40d4-9788-36ebbb799684 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:40.913980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a90a5f-31bf-4c46-ba33-117fdc435d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:40.914891 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a90a5f-31bf-4c46-ba33-117fdc435d1e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:40.915393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a90a5f-31bf-4c46-ba33-117fdc435d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:40.915885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a90a5f-31bf-4c46-ba33-117fdc435d1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:40.917252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a90a5f-31bf-4c46-ba33-117fdc435d1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:40.918915 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a90a5f-31bf-4c46-ba33-117fdc435d1e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:40.920712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 52/105] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:07:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:40.926277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a90a5f-31bf-4c46-ba33-117fdc435d1e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:40.971172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:41.017479 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8b677-c39a-40d4-9788-36ebbb799684 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:41.055791 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:41.061860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5057886-ab6d-4595-af45-19cbacfd055f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:41.062749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5057886-ab6d-4595-af45-19cbacfd055f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:41.063273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5057886-ab6d-4595-af45-19cbacfd055f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:41.063951 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5057886-ab6d-4595-af45-19cbacfd055f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:41.080419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a90a5f-31bf-4c46-ba33-117fdc435d1e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:41.115685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a282ea-7b13-4963-a30d-986114d6f306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:41.116190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a282ea-7b13-4963-a30d-986114d6f306 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:41.116477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a282ea-7b13-4963-a30d-986114d6f306 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:41.116766 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a282ea-7b13-4963-a30d-986114d6f306 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:41.117409 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a282ea-7b13-4963-a30d-986114d6f306 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:41.118858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a282ea-7b13-4963-a30d-986114d6f306 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:41.149773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5057886-ab6d-4595-af45-19cbacfd055f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:41.149950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 53/106] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Oct 6 12:07:40 2022] POST /identity/v3/projects => generated 302 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:07:41.151858 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5057886-ab6d-4595-af45-19cbacfd055f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:41.272125 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:41.320137 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 54/107] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:07:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 12:07:41.326276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a282ea-7b13-4963-a30d-986114d6f306 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:41.462241 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a282ea-7b13-4963-a30d-986114d6f306 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:41.492539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:41.501955 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24917ce6-ae62-47fe-8e50-e9685a6a715c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:41.502666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24917ce6-ae62-47fe-8e50-e9685a6a715c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:41.502951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24917ce6-ae62-47fe-8e50-e9685a6a715c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:41.503227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24917ce6-ae62-47fe-8e50-e9685a6a715c None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:41.504489 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24917ce6-ae62-47fe-8e50-e9685a6a715c None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:41.505994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24917ce6-ae62-47fe-8e50-e9685a6a715c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:41.507640 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 54/108] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:41 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:41.513661 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5057886-ab6d-4595-af45-19cbacfd055f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:41.626293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5057886-ab6d-4595-af45-19cbacfd055f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:41.670015 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219ca4c-6faf-4014-837b-dd9bfdff6297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:41.670638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219ca4c-6faf-4014-837b-dd9bfdff6297 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:41.670927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219ca4c-6faf-4014-837b-dd9bfdff6297 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:41.671208 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219ca4c-6faf-4014-837b-dd9bfdff6297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:41.671975 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219ca4c-6faf-4014-837b-dd9bfdff6297 None admin] RBAC: Adding query filter params (role.id=061fff977e1f4d13a7b5e8e70edc3e92, scope.project.id=b427802e13664086974c8d53f5bfbc88, user.id=1cbd4e62c7a0428ba026ac3c642204c9) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:41.672291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219ca4c-6faf-4014-837b-dd9bfdff6297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:41.673812 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219ca4c-6faf-4014-837b-dd9bfdff6297 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:41.679661 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 55/109] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:07:41 2022] GET /identity/v3/role_assignments?user.id=1cbd4e62c7a0428ba026ac3c642204c9&scope.project.id=b427802e13664086974c8d53f5bfbc88&role.id=061fff977e1f4d13a7b5e8e70edc3e92 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:42.125581 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9d7fb-8a52-4038-92b8-6cef5fd5cd63 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:42.126693 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9d7fb-8a52-4038-92b8-6cef5fd5cd63 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:42.127176 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9d7fb-8a52-4038-92b8-6cef5fd5cd63 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:42.128784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 55/110] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:42.134280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5b01e-fe3e-4e22-972d-639dd572db3a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:42.135138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5b01e-fe3e-4e22-972d-639dd572db3a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:42.136195 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5b01e-fe3e-4e22-972d-639dd572db3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:42.156889 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-bcd5b01e-fe3e-4e22-972d-639dd572db3a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:42.158470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bcd5b01e-fe3e-4e22-972d-639dd572db3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:42.183577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:42.207781 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6663ccc-e3a2-4f77-a0dc-267ebc3016fb None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:42.208394 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6663ccc-e3a2-4f77-a0dc-267ebc3016fb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:42.208893 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6663ccc-e3a2-4f77-a0dc-267ebc3016fb None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:42.210481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 56/111] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:42.220785 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff089264-39bb-4b33-baee-3cce51bdcc03 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:42.221353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff089264-39bb-4b33-baee-3cce51bdcc03 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:42.221826 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff089264-39bb-4b33-baee-3cce51bdcc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:42.250573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ff089264-39bb-4b33-baee-3cce51bdcc03 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:42.252068 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ff089264-39bb-4b33-baee-3cce51bdcc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:42.288654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 56/112] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:42.294640 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828de867-6d59-447c-b0aa-18f704ab9b66 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:42.294934 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828de867-6d59-447c-b0aa-18f704ab9b66 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:42.295274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828de867-6d59-447c-b0aa-18f704ab9b66 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:42.296715 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 57/113] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:42.305367 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518f958-23fe-4e06-97ac-e0ed47887102 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:42.306173 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518f958-23fe-4e06-97ac-e0ed47887102 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:42.308783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518f958-23fe-4e06-97ac-e0ed47887102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:42.336978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f518f958-23fe-4e06-97ac-e0ed47887102 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:42.338514 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f518f958-23fe-4e06-97ac-e0ed47887102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:42.393373 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 57/114] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:42.449947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 58/115] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:07:42.487115 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8d223-d339-45de-9fec-75c225afbd87 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:42.487774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8d223-d339-45de-9fec-75c225afbd87 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:42.488246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8d223-d339-45de-9fec-75c225afbd87 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:42.490043 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 58/116] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:42.495577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f3dcd-4b57-4058-9333-68dd0b351f91 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:42.495872 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f3dcd-4b57-4058-9333-68dd0b351f91 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:42.496150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f3dcd-4b57-4058-9333-68dd0b351f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:42.524456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-bc7f3dcd-4b57-4058-9333-68dd0b351f91 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:42.525654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bc7f3dcd-4b57-4058-9333-68dd0b351f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:42.597188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 59/117] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:42.698758 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8d223-d339-45de-9fec-75c225afbd87 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:42.810362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f3dcd-4b57-4058-9333-68dd0b351f91 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:42.836021 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:42.840871 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8d223-d339-45de-9fec-75c225afbd87 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:42.896545 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:42.913597 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65976c94-5aec-4ce6-aa6d-de891f48ea70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:42.914237 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65976c94-5aec-4ce6-aa6d-de891f48ea70 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:42.914519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65976c94-5aec-4ce6-aa6d-de891f48ea70 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:42.914792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65976c94-5aec-4ce6-aa6d-de891f48ea70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:42.933879 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f3dcd-4b57-4058-9333-68dd0b351f91 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:42.938206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65976c94-5aec-4ce6-aa6d-de891f48ea70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:42.950760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-65976c94-5aec-4ce6-aa6d-de891f48ea70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:07:42.950760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:42.950760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:42.950760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:42.950760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:42.950760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:42.950760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:42.950760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:42.950760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:42.950760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:42.950760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:42.950760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:42.950760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:42.950760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:42.950760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:42.950760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:42.951643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:42.951643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:42.951643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:42.951643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:42.951643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:42.951643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:42.951643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:42.951643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:42.951643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:42.951643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:42.951643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:42.951643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:42.951643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:42.951643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:42.951643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:42.951643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:42.952439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:42.952439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:42.952439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:42.952439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:42.952439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:42.952439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:42.952439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:42.952439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:42.952439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:42.952439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:42.952439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:07:42.952439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:07:42.952439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 59/118] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:07:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:42.953088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65976c94-5aec-4ce6-aa6d-de891f48ea70 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:42.976940 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b9cda-a39c-44d2-8061-178fdbebe45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:42.977875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b9cda-a39c-44d2-8061-178fdbebe45a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:42.978161 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b9cda-a39c-44d2-8061-178fdbebe45a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:42.978437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b9cda-a39c-44d2-8061-178fdbebe45a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:42.992377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b9cda-a39c-44d2-8061-178fdbebe45a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:42.999410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-7a4b9cda-a39c-44d2-8061-178fdbebe45a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:07:42.999410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:42.999410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:42.999410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:42.999410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:42.999410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:42.999410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:42.999410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:42.999410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:42.999410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:42.999410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:42.999410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:42.999410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:42.999410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:42.999410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:42.999410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:43.000365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:43.000365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:43.000365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:43.000365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:43.000365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:43.000365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:43.000365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:43.000365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:43.000365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:43.000365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:43.000365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:43.000365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:43.000365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:43.000365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:43.000365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:43.000365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:43.001139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:43.001139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:43.001139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:43.001139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:43.001139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:43.001139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:43.001139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:43.001139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:43.001139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:43.001139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:43.001139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:07:43.001139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:07:43.002282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 60/119] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:07:43.002282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b9cda-a39c-44d2-8061-178fdbebe45a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:43.069056 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65976c94-5aec-4ce6-aa6d-de891f48ea70 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:43.102319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e9568-e790-4b2c-a60f-e1887d67419f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:43.103036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e9568-e790-4b2c-a60f-e1887d67419f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:43.103391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e9568-e790-4b2c-a60f-e1887d67419f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:43.103792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e9568-e790-4b2c-a60f-e1887d67419f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:43.108004 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:43.114676 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e9568-e790-4b2c-a60f-e1887d67419f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:43.119568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-3d0e9568-e790-4b2c-a60f-e1887d67419f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:07:43.119568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:43.119568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:43.119568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:43.119568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:43.119568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:43.119568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:43.119568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:43.119568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:43.119568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:43.119568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:43.119568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:43.119568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:43.119568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:43.119568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:43.119568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:43.120943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:43.120943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:43.120943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:43.120943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:43.120943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:43.120943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:43.120943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:43.120943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:43.120943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:43.120943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:43.120943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:43.120943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:43.120943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:43.120943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:43.120943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:43.120943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:43.121717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:43.121717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:43.121717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:43.121717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:43.121717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:43.121717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:43.121717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:43.121717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:43.121717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:43.121717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:43.121717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:07:43.121717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:07:43.121717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 60/120] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:07:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:43.122755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e9568-e790-4b2c-a60f-e1887d67419f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:43.142399 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b9cda-a39c-44d2-8061-178fdbebe45a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:43.181895 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11eee8-5ad8-41f6-a28e-a729c0fd7093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:43.183321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11eee8-5ad8-41f6-a28e-a729c0fd7093 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:43.183831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11eee8-5ad8-41f6-a28e-a729c0fd7093 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:43.184319 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11eee8-5ad8-41f6-a28e-a729c0fd7093 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:43.185163 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b11eee8-5ad8-41f6-a28e-a729c0fd7093 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:43.185654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b11eee8-5ad8-41f6-a28e-a729c0fd7093 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:43.187240 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b11eee8-5ad8-41f6-a28e-a729c0fd7093 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:43.187866 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:43.196664 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 61/121] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:43.197654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11eee8-5ad8-41f6-a28e-a729c0fd7093 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:43.262776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e9568-e790-4b2c-a60f-e1887d67419f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:43.297994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd3b52-585a-4812-b3f0-0b6906e9ce71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:43.298969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd3b52-585a-4812-b3f0-0b6906e9ce71 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:43.299443 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd3b52-585a-4812-b3f0-0b6906e9ce71 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:43.299932 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd3b52-585a-4812-b3f0-0b6906e9ce71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:43.310750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd3b52-585a-4812-b3f0-0b6906e9ce71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:43.315854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-d3fd3b52-585a-4812-b3f0-0b6906e9ce71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:07:43.315854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:43.315854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:43.315854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:43.315854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:43.315854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:43.315854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:43.315854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:43.315854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:43.315854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:43.315854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:43.315854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:43.315854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:43.315854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:43.315854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:43.315854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:43.316702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:43.316702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:43.316702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:43.316702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:43.316702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:43.316702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:43.316702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:43.316702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:43.316702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:43.316702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:43.316702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:43.316702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:43.316702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:43.316702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:43.316702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:43.316702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:43.317482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:43.317482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:43.317482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:43.317482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:43.317482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:43.317482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:43.317482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:43.317482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:43.317482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:43.317482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:43.317482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:07:43.317482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:07:43.319287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11eee8-5ad8-41f6-a28e-a729c0fd7093 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:43.323606 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 61/122] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:43.324641 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd3b52-585a-4812-b3f0-0b6906e9ce71 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:43.364168 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb68a8c-8d21-4fd8-a023-89ff00b068ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:43.364793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb68a8c-8d21-4fd8-a023-89ff00b068ba None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:43.365083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb68a8c-8d21-4fd8-a023-89ff00b068ba None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:43.365364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb68a8c-8d21-4fd8-a023-89ff00b068ba None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:43.366040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb68a8c-8d21-4fd8-a023-89ff00b068ba None admin] RBAC: Adding query filter params (name=member) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:43.366349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb68a8c-8d21-4fd8-a023-89ff00b068ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:43.367771 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb68a8c-8d21-4fd8-a023-89ff00b068ba None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:43.374919 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:43.383941 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 62/123] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:07:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:43.385123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb68a8c-8d21-4fd8-a023-89ff00b068ba None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:43.440611 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd3b52-585a-4812-b3f0-0b6906e9ce71 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:43.451253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:43.478203 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fd18f-6efc-4dd2-9c6b-9eec3d7604ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:43.479153 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fd18f-6efc-4dd2-9c6b-9eec3d7604ec None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:43.480194 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fd18f-6efc-4dd2-9c6b-9eec3d7604ec None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:43.480664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fd18f-6efc-4dd2-9c6b-9eec3d7604ec None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:43.481525 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fd18f-6efc-4dd2-9c6b-9eec3d7604ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:43.482024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fd18f-6efc-4dd2-9c6b-9eec3d7604ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:43.483649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fd18f-6efc-4dd2-9c6b-9eec3d7604ec None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:43.492120 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 62/124] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:07:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:43.492840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb68a8c-8d21-4fd8-a023-89ff00b068ba None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:43.493503 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fd18f-6efc-4dd2-9c6b-9eec3d7604ec None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:43.529226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b318369-4c03-4f28-877b-c062d0777c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:43.529905 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b318369-4c03-4f28-877b-c062d0777c39 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:43.531042 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b318369-4c03-4f28-877b-c062d0777c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:43.531042 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b318369-4c03-4f28-877b-c062d0777c39 None admin] PATH_INFO: `/v3/users/50b3556af8274b8a9a59fe39e12271c7` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:43.531971 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b318369-4c03-4f28-877b-c062d0777c39 None admin] RBAC: Authorizing `identity:get_user(user_id=50b3556af8274b8a9a59fe39e12271c7)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:43.533472 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b318369-4c03-4f28-877b-c062d0777c39 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:43.535159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 63/125] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:43 2022] GET /identity/v3/users/50b3556af8274b8a9a59fe39e12271c7 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:43.536017 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b318369-4c03-4f28-877b-c062d0777c39 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:43.598454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fd18f-6efc-4dd2-9c6b-9eec3d7604ec None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:43.603742 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:43.634129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efef583-1db8-4a2a-af85-c4b76098ab76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:43.635080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efef583-1db8-4a2a-af85-c4b76098ab76 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:43.635618 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efef583-1db8-4a2a-af85-c4b76098ab76 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:43.636089 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efef583-1db8-4a2a-af85-c4b76098ab76 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:43.636942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efef583-1db8-4a2a-af85-c4b76098ab76 None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:43.637457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efef583-1db8-4a2a-af85-c4b76098ab76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:43.639316 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efef583-1db8-4a2a-af85-c4b76098ab76 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:43.641404 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b318369-4c03-4f28-877b-c062d0777c39 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:43.648463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 63/126] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:07:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:43.649508 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efef583-1db8-4a2a-af85-c4b76098ab76 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:43.689213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fb24f-bbc6-4f7a-89e5-c63793a6a391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:43.690341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fb24f-bbc6-4f7a-89e5-c63793a6a391 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:43.691090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fb24f-bbc6-4f7a-89e5-c63793a6a391 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:43.691843 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fb24f-bbc6-4f7a-89e5-c63793a6a391 None admin] PATH_INFO: `/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:43.693322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fb24f-bbc6-4f7a-89e5-c63793a6a391 None admin] RBAC: Authorizing `identity:get_user(user_id=02b6e37825904c729c9cf6fd8e1f7dbd)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:43.693768 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:43.696238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fb24f-bbc6-4f7a-89e5-c63793a6a391 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:43.703668 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 64/127] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:43 2022] GET /identity/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:43.704460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fb24f-bbc6-4f7a-89e5-c63793a6a391 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:43.786693 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efef583-1db8-4a2a-af85-c4b76098ab76 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:43.829409 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99316a2-7161-440b-88f3-164817093aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:43.830447 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99316a2-7161-440b-88f3-164817093aac None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:43.830735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99316a2-7161-440b-88f3-164817093aac None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:43.831010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99316a2-7161-440b-88f3-164817093aac None admin] PATH_INFO: `/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:43.832352 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99316a2-7161-440b-88f3-164817093aac None admin] RBAC: Authorizing `identity:get_user(user_id=02b6e37825904c729c9cf6fd8e1f7dbd)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:43.833801 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99316a2-7161-440b-88f3-164817093aac None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:43.834094 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fb24f-bbc6-4f7a-89e5-c63793a6a391 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:43.835547 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 64/128] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:43 2022] GET /identity/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:43.836828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99316a2-7161-440b-88f3-164817093aac None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:43.878870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e774cc3-1b53-459d-bee5-63aac3e56b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:43.880085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e774cc3-1b53-459d-bee5-63aac3e56b17 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:43.881387 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e774cc3-1b53-459d-bee5-63aac3e56b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:43.881867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e774cc3-1b53-459d-bee5-63aac3e56b17 None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:43.883310 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e774cc3-1b53-459d-bee5-63aac3e56b17 None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:43.884987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e774cc3-1b53-459d-bee5-63aac3e56b17 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:43.888139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 65/129] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:43 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:43.889692 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e774cc3-1b53-459d-bee5-63aac3e56b17 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:43.893450 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:43.967142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:43.971533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99316a2-7161-440b-88f3-164817093aac None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:44.007341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e774cc3-1b53-459d-bee5-63aac3e56b17 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:44.018106 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75d232-c408-4afc-bff6-daf29c854d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:44.019020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75d232-c408-4afc-bff6-daf29c854d8e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:44.019583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75d232-c408-4afc-bff6-daf29c854d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:44.020061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75d232-c408-4afc-bff6-daf29c854d8e None admin] PATH_INFO: `/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:44.021379 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75d232-c408-4afc-bff6-daf29c854d8e None admin] RBAC: Authorizing `identity:get_user(user_id=02b6e37825904c729c9cf6fd8e1f7dbd)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:44.023035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75d232-c408-4afc-bff6-daf29c854d8e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:44.024862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 65/130] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:43 2022] GET /identity/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:44.025814 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75d232-c408-4afc-bff6-daf29c854d8e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:44.048824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d3cc5-75c1-467b-9563-7897b0194a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:44.049471 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d3cc5-75c1-467b-9563-7897b0194a53 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:44.049747 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d3cc5-75c1-467b-9563-7897b0194a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:44.050020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d3cc5-75c1-467b-9563-7897b0194a53 None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:44.051191 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d3cc5-75c1-467b-9563-7897b0194a53 None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:44.052820 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d3cc5-75c1-467b-9563-7897b0194a53 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:44.059532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 66/131] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:43 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:44.059532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d3cc5-75c1-467b-9563-7897b0194a53 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:44.140034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:44.150360 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75d232-c408-4afc-bff6-daf29c854d8e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:44.171676 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d3cc5-75c1-467b-9563-7897b0194a53 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:44.189488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d26d3c-e47a-4c1e-bfa7-93fc2fc2fcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:44.190759 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d26d3c-e47a-4c1e-bfa7-93fc2fc2fcb8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:44.191315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d26d3c-e47a-4c1e-bfa7-93fc2fc2fcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:44.191828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d26d3c-e47a-4c1e-bfa7-93fc2fc2fcb8 None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:44.193219 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d26d3c-e47a-4c1e-bfa7-93fc2fc2fcb8 None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:44.195023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d26d3c-e47a-4c1e-bfa7-93fc2fc2fcb8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:44.197160 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 66/132] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:44 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:44.198919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d26d3c-e47a-4c1e-bfa7-93fc2fc2fcb8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:44.208801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506cb00-5c9e-49b5-94ff-717681d00c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:44.209476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506cb00-5c9e-49b5-94ff-717681d00c97 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:44.209759 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506cb00-5c9e-49b5-94ff-717681d00c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:44.210034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506cb00-5c9e-49b5-94ff-717681d00c97 None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88/users/50b3556af8274b8a9a59fe39e12271c7/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:44.212220 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506cb00-5c9e-49b5-94ff-717681d00c97 None admin] RBAC: Authorizing `identity:create_grant(project_id=b427802e13664086974c8d53f5bfbc88, user_id=50b3556af8274b8a9a59fe39e12271c7, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:44.213772 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506cb00-5c9e-49b5-94ff-717681d00c97 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:44.234341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 67/133] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:07:44 2022] PUT /identity/v3/projects/b427802e13664086974c8d53f5bfbc88/users/50b3556af8274b8a9a59fe39e12271c7/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:07:44.235809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506cb00-5c9e-49b5-94ff-717681d00c97 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:44.248628 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:44.325685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d26d3c-e47a-4c1e-bfa7-93fc2fc2fcb8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:44.364090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55adba-c193-49f4-9bbb-a58e72d0c649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:44.364916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55adba-c193-49f4-9bbb-a58e72d0c649 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:44.365298 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55adba-c193-49f4-9bbb-a58e72d0c649 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:44.365681 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55adba-c193-49f4-9bbb-a58e72d0c649 None admin] PATH_INFO: `/v3/projects/f9c2c60f747b4390b69a797a1d5781f7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:44.366988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55adba-c193-49f4-9bbb-a58e72d0c649 None admin] RBAC: Authorizing `identity:get_project(project_id=f9c2c60f747b4390b69a797a1d5781f7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:44.368675 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55adba-c193-49f4-9bbb-a58e72d0c649 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:44.375321 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 67/134] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:44 2022] GET /identity/v3/projects/f9c2c60f747b4390b69a797a1d5781f7 => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:44.375321 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55adba-c193-49f4-9bbb-a58e72d0c649 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:44.382132 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506cb00-5c9e-49b5-94ff-717681d00c97 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:44.421348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:44.430469 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743486c1-0cb7-471b-979f-04f9fed9fbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:44.431821 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743486c1-0cb7-471b-979f-04f9fed9fbd7 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:44.432555 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743486c1-0cb7-471b-979f-04f9fed9fbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:44.432841 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743486c1-0cb7-471b-979f-04f9fed9fbd7 None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88/users/02b6e37825904c729c9cf6fd8e1f7dbd/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:44.435634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743486c1-0cb7-471b-979f-04f9fed9fbd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b427802e13664086974c8d53f5bfbc88, user_id=02b6e37825904c729c9cf6fd8e1f7dbd, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:44.437965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743486c1-0cb7-471b-979f-04f9fed9fbd7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:44.455744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 68/135] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:07:44 2022] PUT /identity/v3/projects/b427802e13664086974c8d53f5bfbc88/users/02b6e37825904c729c9cf6fd8e1f7dbd/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:07:44.457405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e435dd-be75-412a-b046-216138e40f37 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:44.457717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e435dd-be75-412a-b046-216138e40f37 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:44.457975 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e435dd-be75-412a-b046-216138e40f37 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:44.459577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 69/136] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:44.460143 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e435dd-be75-412a-b046-216138e40f37 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:44.509862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55adba-c193-49f4-9bbb-a58e72d0c649 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:44.513350 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:44.553037 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700ea83-ee1c-4429-9f4f-4b6257e874b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:44.553742 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700ea83-ee1c-4429-9f4f-4b6257e874b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:44.554027 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700ea83-ee1c-4429-9f4f-4b6257e874b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:44.554305 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700ea83-ee1c-4429-9f4f-4b6257e874b4 None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88/users/02b6e37825904c729c9cf6fd8e1f7dbd/roles/2c11a95c1ecf4988a95aad8d96ea56e9` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:44.557951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700ea83-ee1c-4429-9f4f-4b6257e874b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=b427802e13664086974c8d53f5bfbc88, user_id=02b6e37825904c729c9cf6fd8e1f7dbd, role_id=2c11a95c1ecf4988a95aad8d96ea56e9)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:44.559770 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700ea83-ee1c-4429-9f4f-4b6257e874b4 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:44.580226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 68/137] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:07:44 2022] PUT /identity/v3/projects/b427802e13664086974c8d53f5bfbc88/users/02b6e37825904c729c9cf6fd8e1f7dbd/roles/2c11a95c1ecf4988a95aad8d96ea56e9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:07:44.581939 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0ddb9-c3b0-4f42-8688-bf8220a2da6c None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:44.582232 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0ddb9-c3b0-4f42-8688-bf8220a2da6c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:44.582598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0ddb9-c3b0-4f42-8688-bf8220a2da6c None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:44.584947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 69/138] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:44.588139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354dc822-b6c4-48d8-8c24-df9555fcbfdd None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:44.588426 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354dc822-b6c4-48d8-8c24-df9555fcbfdd None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:44.589224 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354dc822-b6c4-48d8-8c24-df9555fcbfdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:44.604571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e435dd-be75-412a-b046-216138e40f37 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:44.614147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-354dc822-b6c4-48d8-8c24-df9555fcbfdd None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:44.615977 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-354dc822-b6c4-48d8-8c24-df9555fcbfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:44.648961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddbba8-736f-45f0-80a9-4f23db10f29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:44.649747 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddbba8-736f-45f0-80a9-4f23db10f29e None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:44.650082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddbba8-736f-45f0-80a9-4f23db10f29e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:44.650410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddbba8-736f-45f0-80a9-4f23db10f29e None admin] PATH_INFO: `/v3/projects/f9c2c60f747b4390b69a797a1d5781f7/users/02b6e37825904c729c9cf6fd8e1f7dbd/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:44.653596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddbba8-736f-45f0-80a9-4f23db10f29e None admin] RBAC: Authorizing `identity:create_grant(project_id=f9c2c60f747b4390b69a797a1d5781f7, user_id=02b6e37825904c729c9cf6fd8e1f7dbd, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:44.656349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddbba8-736f-45f0-80a9-4f23db10f29e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:44.691439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 70/139] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:07:44 2022] PUT /identity/v3/projects/f9c2c60f747b4390b69a797a1d5781f7/users/02b6e37825904c729c9cf6fd8e1f7dbd/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:07:44.701242 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36947918-ce47-4bc3-9569-ff87cb46c89b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:44.701598 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36947918-ce47-4bc3-9569-ff87cb46c89b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:44.701875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36947918-ce47-4bc3-9569-ff87cb46c89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:44.717261 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 70/140] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:44.729096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-36947918-ce47-4bc3-9569-ff87cb46c89b None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:44.730442 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-36947918-ce47-4bc3-9569-ff87cb46c89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:44.801473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:44.828029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 71/141] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:45.134236 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354dc822-b6c4-48d8-8c24-df9555fcbfdd None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:45.194533 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36947918-ce47-4bc3-9569-ff87cb46c89b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:45.268822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:45.277346 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354dc822-b6c4-48d8-8c24-df9555fcbfdd None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:45.323360 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f5423-65e0-4e62-9950-a2dbb1d8a3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:45.324039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f5423-65e0-4e62-9950-a2dbb1d8a3e1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:45.324325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f5423-65e0-4e62-9950-a2dbb1d8a3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:45.324614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f5423-65e0-4e62-9950-a2dbb1d8a3e1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:45.345641 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f5423-65e0-4e62-9950-a2dbb1d8a3e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:45.350965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-368f5423-65e0-4e62-9950-a2dbb1d8a3e1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:07:45.350965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:45.350965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:45.350965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:45.350965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:45.350965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:45.350965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:45.350965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:45.350965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:45.350965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:45.350965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:45.350965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:45.350965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:45.350965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:45.350965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:45.350965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:45.351928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:45.351928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:45.351928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:45.351928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:45.351928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:45.351928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:45.351928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:45.351928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:45.351928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:45.351928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:45.351928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:45.351928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:45.351928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:45.351928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:45.351928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:45.351928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:45.352696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:45.352696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:45.352696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:45.352696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:45.352696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:45.352696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:45.352696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:45.352696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:45.352696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:45.352696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:45.352696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:07:45.352696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:07:45.353293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36947918-ce47-4bc3-9569-ff87cb46c89b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:45.360520 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 71/142] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:45.366305 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f5423-65e0-4e62-9950-a2dbb1d8a3e1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:45.388993 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50975f65-74e5-47f2-9dc7-57fedd7a67e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:45.389552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50975f65-74e5-47f2-9dc7-57fedd7a67e3 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:45.389834 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50975f65-74e5-47f2-9dc7-57fedd7a67e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:45.390104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50975f65-74e5-47f2-9dc7-57fedd7a67e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:45.391213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50975f65-74e5-47f2-9dc7-57fedd7a67e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:45.392731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50975f65-74e5-47f2-9dc7-57fedd7a67e3 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:45.394698 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 72/143] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:07:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:45.402570 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50975f65-74e5-47f2-9dc7-57fedd7a67e3 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:45.491391 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:45.586040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f5423-65e0-4e62-9950-a2dbb1d8a3e1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:45.600234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50975f65-74e5-47f2-9dc7-57fedd7a67e3 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:45.656416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce01dfb-9467-4a06-80a8-69edd3b5d7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:45.656950 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce01dfb-9467-4a06-80a8-69edd3b5d7e8 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:45.657250 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce01dfb-9467-4a06-80a8-69edd3b5d7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:45.657522 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce01dfb-9467-4a06-80a8-69edd3b5d7e8 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:45.658180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce01dfb-9467-4a06-80a8-69edd3b5d7e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:45.660354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce01dfb-9467-4a06-80a8-69edd3b5d7e8 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:45.665833 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefe1b0-1bfb-4a20-8366-74cbfba7f551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:45.666449 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefe1b0-1bfb-4a20-8366-74cbfba7f551 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:45.666728 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefe1b0-1bfb-4a20-8366-74cbfba7f551 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:45.667537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefe1b0-1bfb-4a20-8366-74cbfba7f551 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:45.668215 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcefe1b0-1bfb-4a20-8366-74cbfba7f551 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:45.668513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcefe1b0-1bfb-4a20-8366-74cbfba7f551 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:45.669897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcefe1b0-1bfb-4a20-8366-74cbfba7f551 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:45.687961 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 72/144] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:07:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:07:45.693791 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefe1b0-1bfb-4a20-8366-74cbfba7f551 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:45.728676 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 73/145] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:07:45 2022] POST /identity/v3/users => generated 283 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:07:45.740233 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:45.858403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefe1b0-1bfb-4a20-8366-74cbfba7f551 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:45.906832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f3c9e-8d7e-458f-a960-dad1e4b57d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:45.907404 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f3c9e-8d7e-458f-a960-dad1e4b57d68 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:45.907713 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f3c9e-8d7e-458f-a960-dad1e4b57d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:45.907990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f3c9e-8d7e-458f-a960-dad1e4b57d68 None admin] PATH_INFO: `/v3/users/1cbd4e62c7a0428ba026ac3c642204c9` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:45.909167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f3c9e-8d7e-458f-a960-dad1e4b57d68 None admin] RBAC: Authorizing `identity:get_user(user_id=1cbd4e62c7a0428ba026ac3c642204c9)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:45.910637 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f3c9e-8d7e-458f-a960-dad1e4b57d68 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:45.912282 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 73/146] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:45 2022] GET /identity/v3/users/1cbd4e62c7a0428ba026ac3c642204c9 => generated 289 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:07:45.918670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce01dfb-9467-4a06-80a8-69edd3b5d7e8 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:46.057106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce01dfb-9467-4a06-80a8-69edd3b5d7e8 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:46.060133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:46.101777 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d28187-c95a-4e5b-89d4-ddfa0556e1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:46.102424 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d28187-c95a-4e5b-89d4-ddfa0556e1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:46.102705 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d28187-c95a-4e5b-89d4-ddfa0556e1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:46.102978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d28187-c95a-4e5b-89d4-ddfa0556e1b5 None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:46.106123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d28187-c95a-4e5b-89d4-ddfa0556e1b5 None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:46.107605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d28187-c95a-4e5b-89d4-ddfa0556e1b5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:46.109557 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 74/147] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:45 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:46.114945 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f3c9e-8d7e-458f-a960-dad1e4b57d68 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:46.223536 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:46.250632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f3c9e-8d7e-458f-a960-dad1e4b57d68 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:46.293273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00acaca-12a2-4f76-a4ca-d6d71cb0dc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:46.294023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00acaca-12a2-4f76-a4ca-d6d71cb0dc5e None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:46.294263 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00acaca-12a2-4f76-a4ca-d6d71cb0dc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:46.294540 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00acaca-12a2-4f76-a4ca-d6d71cb0dc5e None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88/users/1cbd4e62c7a0428ba026ac3c642204c9/roles/061fff977e1f4d13a7b5e8e70edc3e92` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:46.296485 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00acaca-12a2-4f76-a4ca-d6d71cb0dc5e None admin] RBAC: Authorizing `identity:create_grant(project_id=b427802e13664086974c8d53f5bfbc88, user_id=1cbd4e62c7a0428ba026ac3c642204c9, role_id=061fff977e1f4d13a7b5e8e70edc3e92)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:46.297972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00acaca-12a2-4f76-a4ca-d6d71cb0dc5e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:46.317435 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 74/148] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:07:46 2022] PUT /identity/v3/projects/b427802e13664086974c8d53f5bfbc88/users/1cbd4e62c7a0428ba026ac3c642204c9/roles/061fff977e1f4d13a7b5e8e70edc3e92 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:07:47.010196 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fc7e9-b899-4e29-a35a-9f950ed07767 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:47.010757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fc7e9-b899-4e29-a35a-9f950ed07767 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:47.010757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fc7e9-b899-4e29-a35a-9f950ed07767 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:47.012143 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 75/149] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:47.017616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9c390-96d3-4465-bbbc-9962d572053e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:47.017899 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9c390-96d3-4465-bbbc-9962d572053e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:47.018188 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9c390-96d3-4465-bbbc-9962d572053e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:47.040788 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a6e9c390-96d3-4465-bbbc-9962d572053e None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:47.042114 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a6e9c390-96d3-4465-bbbc-9962d572053e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:47.118511 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 75/150] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:47.333335 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c00d0-4485-4093-9507-ea67fc5ed107 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:47.333954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c00d0-4485-4093-9507-ea67fc5ed107 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:47.334445 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c00d0-4485-4093-9507-ea67fc5ed107 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:47.336289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 76/151] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:47.341675 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0c581-3f7f-4fdd-97d2-af8b06aa73de None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:47.342553 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0c581-3f7f-4fdd-97d2-af8b06aa73de None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:47.343022 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0c581-3f7f-4fdd-97d2-af8b06aa73de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:47.367321 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-08a0c581-3f7f-4fdd-97d2-af8b06aa73de None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:47.368848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-08a0c581-3f7f-4fdd-97d2-af8b06aa73de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:47.468057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 76/152] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:47.623077 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c00d0-4485-4093-9507-ea67fc5ed107 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:47.753940 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c00d0-4485-4093-9507-ea67fc5ed107 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:47.803072 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645a1d2-b9aa-49a0-89d4-0fd158984a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:47.804108 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645a1d2-b9aa-49a0-89d4-0fd158984a80 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:47.804586 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645a1d2-b9aa-49a0-89d4-0fd158984a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:47.805061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645a1d2-b9aa-49a0-89d4-0fd158984a80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:47.820799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645a1d2-b9aa-49a0-89d4-0fd158984a80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:47.826104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-c645a1d2-b9aa-49a0-89d4-0fd158984a80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:07:47.826104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:47.826104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:47.826104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:47.826104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:47.826104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:47.826104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:47.826104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:47.826104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:47.826104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:47.826104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:47.826104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:47.826104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:47.826104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:47.826104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:47.826104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:47.827030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:47.827030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:47.827030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:47.827030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:47.827030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:47.827030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:47.827030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:47.827030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:47.827030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:47.827030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:47.827030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:47.827030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:47.827030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:47.827030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:47.827030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:47.827030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:47.827862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:47.827862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:47.827862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:47.827862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:47.827862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:47.827862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:47.827862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:47.827862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:47.827862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:47.827862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:47.827862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:07:47.827862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:07:47.829803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 77/153] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:07:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:47.835763 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0c581-3f7f-4fdd-97d2-af8b06aa73de None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:47.839614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:47.949694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645a1d2-b9aa-49a0-89d4-0fd158984a80 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:47.970861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0c581-3f7f-4fdd-97d2-af8b06aa73de None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:47.992267 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:48.042854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ae432-1eff-4851-b7d1-572aabbfcdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:48.043498 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ae432-1eff-4851-b7d1-572aabbfcdb2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:48.043830 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ae432-1eff-4851-b7d1-572aabbfcdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:48.044106 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ae432-1eff-4851-b7d1-572aabbfcdb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:48.044796 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ae432-1eff-4851-b7d1-572aabbfcdb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:48.045101 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ae432-1eff-4851-b7d1-572aabbfcdb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:48.046627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ae432-1eff-4851-b7d1-572aabbfcdb2 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:48.064323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 77/154] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:48.070018 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ae432-1eff-4851-b7d1-572aabbfcdb2 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:48.075801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645a1d2-b9aa-49a0-89d4-0fd158984a80 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:48.122094 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a074ee-2506-498e-b8c1-4c0eddec118f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:48.122976 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a074ee-2506-498e-b8c1-4c0eddec118f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:48.123401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a074ee-2506-498e-b8c1-4c0eddec118f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:48.123869 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a074ee-2506-498e-b8c1-4c0eddec118f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:48.149957 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a074ee-2506-498e-b8c1-4c0eddec118f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:48.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-a0a074ee-2506-498e-b8c1-4c0eddec118f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:07:48.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:48.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:48.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:48.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:48.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:48.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:48.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:48.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:48.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:48.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:48.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:48.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:48.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:48.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:48.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:48.159937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:48.159937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:48.159937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:48.159937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:48.159937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:48.159937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:48.159937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:48.159937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:48.159937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:48.159937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:48.159937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:48.159937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:48.159937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:48.159937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:48.159937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:48.159937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:48.161106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:48.161106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:48.161106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:48.161106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:48.161106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:48.161106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:48.161106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:48.161106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:48.161106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:48.161106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:48.161106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:07:48.161106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:07:48.164725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 78/155] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:48.171268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a074ee-2506-498e-b8c1-4c0eddec118f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:48.208710 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:48.259801 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ae432-1eff-4851-b7d1-572aabbfcdb2 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:48.340322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e10159-698f-4b09-8b01-f65f8901d995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:48.341159 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e10159-698f-4b09-8b01-f65f8901d995 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:48.341445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e10159-698f-4b09-8b01-f65f8901d995 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:48.341726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e10159-698f-4b09-8b01-f65f8901d995 None admin] PATH_INFO: `/v3/users/50b3556af8274b8a9a59fe39e12271c7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:48.343081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e10159-698f-4b09-8b01-f65f8901d995 None admin] RBAC: Authorizing `identity:get_user(user_id=50b3556af8274b8a9a59fe39e12271c7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:48.344677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e10159-698f-4b09-8b01-f65f8901d995 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:48.346461 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 78/156] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:48 2022] GET /identity/v3/users/50b3556af8274b8a9a59fe39e12271c7 => generated 247 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:07:48.352111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b13110-941b-49e8-b5a6-9377b43b8419 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:48.352494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b13110-941b-49e8-b5a6-9377b43b8419 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:48.352779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b13110-941b-49e8-b5a6-9377b43b8419 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:48.354464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 79/157] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:48.355529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b13110-941b-49e8-b5a6-9377b43b8419 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:48.380141 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:48.395227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a074ee-2506-498e-b8c1-4c0eddec118f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:48.473717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c1661-7783-42bc-b7d0-d463e6120f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:48.474685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c1661-7783-42bc-b7d0-d463e6120f85 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:48.475390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c1661-7783-42bc-b7d0-d463e6120f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:48.476762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c1661-7783-42bc-b7d0-d463e6120f85 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:48.477639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c1661-7783-42bc-b7d0-d463e6120f85 None admin] RBAC: Adding query filter params (name=member) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:48.478331 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c1661-7783-42bc-b7d0-d463e6120f85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:48.482057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c1661-7783-42bc-b7d0-d463e6120f85 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:48.500855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 79/158] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:07:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:48.513254 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9097b-dfc0-4061-8e96-c1ba8248fefc None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:48.513826 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9097b-dfc0-4061-8e96-c1ba8248fefc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:48.514304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9097b-dfc0-4061-8e96-c1ba8248fefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:48.547771 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-9bd9097b-dfc0-4061-8e96-c1ba8248fefc None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:48.551290 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9bd9097b-dfc0-4061-8e96-c1ba8248fefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:48.610847 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b13110-941b-49e8-b5a6-9377b43b8419 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:48.693804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035b99c-2af1-4ba7-8d10-ed29d2a778ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:48.695006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035b99c-2af1-4ba7-8d10-ed29d2a778ae None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:48.696189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035b99c-2af1-4ba7-8d10-ed29d2a778ae None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:48.696801 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035b99c-2af1-4ba7-8d10-ed29d2a778ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:48.708077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:48.721127 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 80/159] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:48.724073 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9097b-dfc0-4061-8e96-c1ba8248fefc None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:48.847597 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035b99c-2af1-4ba7-8d10-ed29d2a778ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:48.851019 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035b99c-2af1-4ba7-8d10-ed29d2a778ae None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:48.906695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:48.937646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9097b-dfc0-4061-8e96-c1ba8248fefc None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:49.000655 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63845129-b57a-4590-92f2-a25caeae429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:49.001627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63845129-b57a-4590-92f2-a25caeae429a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:49.002128 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63845129-b57a-4590-92f2-a25caeae429a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:49.002604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63845129-b57a-4590-92f2-a25caeae429a None admin] PATH_INFO: `/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:49.003998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63845129-b57a-4590-92f2-a25caeae429a None admin] RBAC: Authorizing `identity:get_user(user_id=02b6e37825904c729c9cf6fd8e1f7dbd)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:49.005647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63845129-b57a-4590-92f2-a25caeae429a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:49.007526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 81/160] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:48 2022] GET /identity/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:49.013463 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264802aa-efbb-4ae6-83bb-5f469716d95b None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:49.013850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264802aa-efbb-4ae6-83bb-5f469716d95b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:49.014227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264802aa-efbb-4ae6-83bb-5f469716d95b None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:49.015684 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 82/161] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:49.016596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264802aa-efbb-4ae6-83bb-5f469716d95b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:49.052954 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 80/162] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:07:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:07:49.055591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40988bf5-2e6d-4e9f-8732-475e873b849c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:49.055857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40988bf5-2e6d-4e9f-8732-475e873b849c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:49.056130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40988bf5-2e6d-4e9f-8732-475e873b849c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:49.089069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-40988bf5-2e6d-4e9f-8732-475e873b849c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:49.090331 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-40988bf5-2e6d-4e9f-8732-475e873b849c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:49.136751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:49.167637 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264802aa-efbb-4ae6-83bb-5f469716d95b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:49.204866 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff774ddf-472d-477b-9488-d9c7e2bd04a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:49.205489 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff774ddf-472d-477b-9488-d9c7e2bd04a0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:49.205772 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff774ddf-472d-477b-9488-d9c7e2bd04a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:49.206048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff774ddf-472d-477b-9488-d9c7e2bd04a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:49.230048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 81/163] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:49.230728 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40988bf5-2e6d-4e9f-8732-475e873b849c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:49.234523 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:49.319235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff774ddf-472d-477b-9488-d9c7e2bd04a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:49.320908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff774ddf-472d-477b-9488-d9c7e2bd04a0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:49.388628 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40988bf5-2e6d-4e9f-8732-475e873b849c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:49.442589 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0139a-6818-4282-b6b0-f231cb45a6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:49.443240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0139a-6818-4282-b6b0-f231cb45a6fa None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:49.443535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0139a-6818-4282-b6b0-f231cb45a6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:49.444243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0139a-6818-4282-b6b0-f231cb45a6fa None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:49.445481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0139a-6818-4282-b6b0-f231cb45a6fa None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:49.447004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0139a-6818-4282-b6b0-f231cb45a6fa None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:49.449292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 82/164] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:49 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:49.450072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0139a-6818-4282-b6b0-f231cb45a6fa None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:49.496116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 83/165] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:07:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:07:49.496983 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff774ddf-472d-477b-9488-d9c7e2bd04a0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:49.504378 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:49.601772 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0139a-6818-4282-b6b0-f231cb45a6fa None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:49.619620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:49.654947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243866b-e36e-4916-8fbe-5cc638feabed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:49.655868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243866b-e36e-4916-8fbe-5cc638feabed None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:49.656150 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243866b-e36e-4916-8fbe-5cc638feabed None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:49.656422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243866b-e36e-4916-8fbe-5cc638feabed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:49.671168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243866b-e36e-4916-8fbe-5cc638feabed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:49.676142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-5243866b-e36e-4916-8fbe-5cc638feabed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:07:49.676142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:49.676142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:49.676142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:49.676142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:49.676142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:49.676142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:49.676142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:49.676142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:49.676142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:49.676142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:49.676142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:49.676142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:49.676142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:49.676142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:49.676142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:49.677017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:49.677017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:49.677017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:49.677017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:49.677017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:49.677017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:49.677017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:49.677017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:49.677017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:49.677017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:49.677017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:49.677017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:49.677017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:49.677017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:49.677017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:49.677017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:49.678051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:49.678051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:49.678051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:49.678051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:49.678051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:49.678051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:49.678051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:49.678051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:49.678051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:49.678051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:49.678051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:07:49.678051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:07:49.678051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 83/166] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:07:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:49.678051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243866b-e36e-4916-8fbe-5cc638feabed None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:49.747639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff774ddf-472d-477b-9488-d9c7e2bd04a0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:49.805652 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:49.810717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ee0b0-fbd7-4450-b925-51e061b9941f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:49.811597 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ee0b0-fbd7-4450-b925-51e061b9941f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:49.811597 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ee0b0-fbd7-4450-b925-51e061b9941f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:49.811875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ee0b0-fbd7-4450-b925-51e061b9941f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:49.812596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ee0b0-fbd7-4450-b925-51e061b9941f None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=b427802e13664086974c8d53f5bfbc88, user.id=50b3556af8274b8a9a59fe39e12271c7) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:49.812898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ee0b0-fbd7-4450-b925-51e061b9941f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:49.814339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ee0b0-fbd7-4450-b925-51e061b9941f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:49.819315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243866b-e36e-4916-8fbe-5cc638feabed None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:49.835416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 84/167] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:07:49 2022] GET /identity/v3/role_assignments?user.id=50b3556af8274b8a9a59fe39e12271c7&scope.project.id=b427802e13664086974c8d53f5bfbc88&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 608 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:49.836297 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ee0b0-fbd7-4450-b925-51e061b9941f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:49.875135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb48e9-5b93-4f33-a863-6b46e6c2e836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:49.875830 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb48e9-5b93-4f33-a863-6b46e6c2e836 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:49.876112 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb48e9-5b93-4f33-a863-6b46e6c2e836 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:49.876390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb48e9-5b93-4f33-a863-6b46e6c2e836 None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:49.877532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb48e9-5b93-4f33-a863-6b46e6c2e836 None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:49.878991 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb48e9-5b93-4f33-a863-6b46e6c2e836 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:49.880825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 84/168] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:49 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:49.888750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb48e9-5b93-4f33-a863-6b46e6c2e836 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:50.050014 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ee0b0-fbd7-4450-b925-51e061b9941f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:50.076004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb48e9-5b93-4f33-a863-6b46e6c2e836 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:50.090634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:50.127831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d6d7c-4901-47f6-babb-7eedc0d948d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:50.129642 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d6d7c-4901-47f6-babb-7eedc0d948d0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:50.130685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d6d7c-4901-47f6-babb-7eedc0d948d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:50.130968 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d6d7c-4901-47f6-babb-7eedc0d948d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:50.132215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d6d7c-4901-47f6-babb-7eedc0d948d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:50.132526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d6d7c-4901-47f6-babb-7eedc0d948d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:50.134120 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d6d7c-4901-47f6-babb-7eedc0d948d0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:50.143310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a8dca-44a8-4534-aef1-ea23cbb2ad3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:50.143977 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a8dca-44a8-4534-aef1-ea23cbb2ad3a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:50.144253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a8dca-44a8-4534-aef1-ea23cbb2ad3a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:50.144513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a8dca-44a8-4534-aef1-ea23cbb2ad3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:50.150091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 85/169] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:07:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:50.157777 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d6d7c-4901-47f6-babb-7eedc0d948d0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:50.172643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a8dca-44a8-4534-aef1-ea23cbb2ad3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:50.179987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-eb4a8dca-44a8-4534-aef1-ea23cbb2ad3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:07:50.179987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:50.179987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:50.179987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:50.179987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:50.179987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:50.179987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:50.179987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:50.179987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:50.179987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:50.179987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:50.179987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:50.179987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:50.179987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:50.179987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:50.179987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:50.180929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:50.180929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:50.180929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:50.180929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:50.180929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:50.180929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:50.180929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:50.180929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:50.180929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:50.180929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:50.180929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:50.180929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:50.180929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:50.180929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:50.180929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:50.180929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:50.182408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:50.182408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:50.182408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:50.182408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:50.182408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:50.182408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:50.182408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:50.182408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:50.182408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:50.182408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:50.182408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:07:50.182408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:07:50.182408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 85/170] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:50.183118 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a8dca-44a8-4534-aef1-ea23cbb2ad3a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:50.221604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:50.352482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d6d7c-4901-47f6-babb-7eedc0d948d0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:50.372915 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a8dca-44a8-4534-aef1-ea23cbb2ad3a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:50.413694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b25e0e-50c6-44f3-8e03-599e55f74799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:50.414454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b25e0e-50c6-44f3-8e03-599e55f74799 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:50.414692 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b25e0e-50c6-44f3-8e03-599e55f74799 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:50.414851 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b25e0e-50c6-44f3-8e03-599e55f74799 None admin] PATH_INFO: `/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:50.416003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b25e0e-50c6-44f3-8e03-599e55f74799 None admin] RBAC: Authorizing `identity:get_user(user_id=02b6e37825904c729c9cf6fd8e1f7dbd)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:50.418556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b25e0e-50c6-44f3-8e03-599e55f74799 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:50.422466 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 86/171] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:50 2022] GET /identity/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd => generated 275 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:50.427887 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b25e0e-50c6-44f3-8e03-599e55f74799 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:50.448819 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e1261-ae80-49de-90e1-d009a89db8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:50.449437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e1261-ae80-49de-90e1-d009a89db8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:50.449716 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e1261-ae80-49de-90e1-d009a89db8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:50.449985 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e1261-ae80-49de-90e1-d009a89db8e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:50.450727 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e1261-ae80-49de-90e1-d009a89db8e9 None admin] RBAC: Adding query filter params (role.id=bbea734b8a704d7ca8ec61bd45536cbe, scope.project.id=b427802e13664086974c8d53f5bfbc88, user.id=02b6e37825904c729c9cf6fd8e1f7dbd) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:50.451032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e1261-ae80-49de-90e1-d009a89db8e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:50.452501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e1261-ae80-49de-90e1-d009a89db8e9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:50.472969 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 86/172] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:07:50 2022] GET /identity/v3/role_assignments?user.id=02b6e37825904c729c9cf6fd8e1f7dbd&scope.project.id=b427802e13664086974c8d53f5bfbc88&role.id=bbea734b8a704d7ca8ec61bd45536cbe => generated 608 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:50.473842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e1261-ae80-49de-90e1-d009a89db8e9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:50.555880 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:50.569909 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:50.598387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b25e0e-50c6-44f3-8e03-599e55f74799 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:50.657450 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef434421-54ae-4241-9d88-a0c821f1a9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:50.658101 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef434421-54ae-4241-9d88-a0c821f1a9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:50.658384 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef434421-54ae-4241-9d88-a0c821f1a9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:50.658663 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef434421-54ae-4241-9d88-a0c821f1a9b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:50.659364 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef434421-54ae-4241-9d88-a0c821f1a9b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:50.659738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef434421-54ae-4241-9d88-a0c821f1a9b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:50.661201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef434421-54ae-4241-9d88-a0c821f1a9b8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:50.669695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e1261-ae80-49de-90e1-d009a89db8e9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:50.677623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 87/173] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:07:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:50.685831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdc4a7-d28f-4cb5-991a-275b36bd813a None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:50.686345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdc4a7-d28f-4cb5-991a-275b36bd813a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:50.686859 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdc4a7-d28f-4cb5-991a-275b36bd813a None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:50.688907 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 88/174] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:50 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:50.689750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdc4a7-d28f-4cb5-991a-275b36bd813a None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:50.744351 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158cc06-3b91-40e4-8617-1e7f31e2def2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:50.745519 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158cc06-3b91-40e4-8617-1e7f31e2def2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:50.745793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158cc06-3b91-40e4-8617-1e7f31e2def2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:50.746061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158cc06-3b91-40e4-8617-1e7f31e2def2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:50.848088 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158cc06-3b91-40e4-8617-1e7f31e2def2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:50.849569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158cc06-3b91-40e4-8617-1e7f31e2def2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:50.895605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:50.920604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdc4a7-d28f-4cb5-991a-275b36bd813a None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:50.958037 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 87/175] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:07:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:07:50.959550 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b1784-dfb1-43c4-8f78-88c58e5cc22d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:50.960734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b1784-dfb1-43c4-8f78-88c58e5cc22d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:50.961010 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b1784-dfb1-43c4-8f78-88c58e5cc22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:50.966164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5bfaa-7ee0-4655-9430-057bdfc54a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:50.966809 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5bfaa-7ee0-4655-9430-057bdfc54a3b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:50.967086 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5bfaa-7ee0-4655-9430-057bdfc54a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:50.967353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5bfaa-7ee0-4655-9430-057bdfc54a3b None admin] PATH_INFO: `/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:50.968753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5bfaa-7ee0-4655-9430-057bdfc54a3b None admin] RBAC: Authorizing `identity:get_user(user_id=02b6e37825904c729c9cf6fd8e1f7dbd)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:50.971112 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5bfaa-7ee0-4655-9430-057bdfc54a3b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:50.973039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 89/176] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:50 2022] GET /identity/v3/users/02b6e37825904c729c9cf6fd8e1f7dbd => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:50.974197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d4377-adb0-40fb-a851-c127d7e899cb None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:50.974475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d4377-adb0-40fb-a851-c127d7e899cb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:50.974745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d4377-adb0-40fb-a851-c127d7e899cb None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:50.978928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 90/177] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:50.978928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ef4b7-a17a-4e75-b740-10df3aa2e8f8 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:50.978928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ef4b7-a17a-4e75-b740-10df3aa2e8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:50.978928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ef4b7-a17a-4e75-b740-10df3aa2e8f8 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:50.979907 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 91/178] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:50.980413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ef4b7-a17a-4e75-b740-10df3aa2e8f8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:50.988405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f33b1784-dfb1-43c4-8f78-88c58e5cc22d None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:50.989863 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f33b1784-dfb1-43c4-8f78-88c58e5cc22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:51.008930 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:51.070818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 88/179] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:51.071049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b1784-dfb1-43c4-8f78-88c58e5cc22d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:51.103738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ef4b7-a17a-4e75-b740-10df3aa2e8f8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:51.140792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00b26d-e918-4c0f-89f9-2ab656818fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:51.141618 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00b26d-e918-4c0f-89f9-2ab656818fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:51.141990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00b26d-e918-4c0f-89f9-2ab656818fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:51.142385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00b26d-e918-4c0f-89f9-2ab656818fe0 None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:51.143701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df00b26d-e918-4c0f-89f9-2ab656818fe0 None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:51.145480 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df00b26d-e918-4c0f-89f9-2ab656818fe0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:51.155493 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 92/180] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:50 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:51.157186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c86408-49f9-4ef7-b211-baaa8949f97b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:51.157563 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c86408-49f9-4ef7-b211-baaa8949f97b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:51.158007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c86408-49f9-4ef7-b211-baaa8949f97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:51.158660 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:51.181649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d8c86408-49f9-4ef7-b211-baaa8949f97b None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:51.183962 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d8c86408-49f9-4ef7-b211-baaa8949f97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:51.200608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b1784-dfb1-43c4-8f78-88c58e5cc22d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:51.245134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a49cc-e12b-4595-b1b6-071b27a7f526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:51.245767 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a49cc-e12b-4595-b1b6-071b27a7f526 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:51.246049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a49cc-e12b-4595-b1b6-071b27a7f526 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:51.246703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a49cc-e12b-4595-b1b6-071b27a7f526 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:51.269683 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 93/181] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:51.271683 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8554861-33e3-40fe-a663-f628f0d53a2f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:51.272189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8554861-33e3-40fe-a663-f628f0d53a2f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:51.272661 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8554861-33e3-40fe-a663-f628f0d53a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:51.294748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f8554861-33e3-40fe-a663-f628f0d53a2f None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:51.296470 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f8554861-33e3-40fe-a663-f628f0d53a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:51.317936 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:51.330812 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a49cc-e12b-4595-b1b6-071b27a7f526 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:51.333115 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a49cc-e12b-4595-b1b6-071b27a7f526 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:51.402899 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 94/182] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:51.403623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8554861-33e3-40fe-a663-f628f0d53a2f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:51.437182 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:51.472009 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 89/183] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:07:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:07:51.472994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a49cc-e12b-4595-b1b6-071b27a7f526 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:51.549489 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8554861-33e3-40fe-a663-f628f0d53a2f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:51.579719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a49cc-e12b-4595-b1b6-071b27a7f526 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:51.587253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0877b9-6995-4936-9a99-279f62ebdcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:51.588080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0877b9-6995-4936-9a99-279f62ebdcc1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:51.588455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0877b9-6995-4936-9a99-279f62ebdcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:51.588878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0877b9-6995-4936-9a99-279f62ebdcc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:51.589799 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0877b9-6995-4936-9a99-279f62ebdcc1 None admin] RBAC: Adding query filter params (role.id=2c11a95c1ecf4988a95aad8d96ea56e9, scope.project.id=b427802e13664086974c8d53f5bfbc88, user.id=02b6e37825904c729c9cf6fd8e1f7dbd) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:51.590172 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0877b9-6995-4936-9a99-279f62ebdcc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:51.591809 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0877b9-6995-4936-9a99-279f62ebdcc1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:51.598421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 95/184] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:07:51 2022] GET /identity/v3/role_assignments?user.id=02b6e37825904c729c9cf6fd8e1f7dbd&scope.project.id=b427802e13664086974c8d53f5bfbc88&role.id=2c11a95c1ecf4988a95aad8d96ea56e9 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:51.599994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0877b9-6995-4936-9a99-279f62ebdcc1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:51.614932 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:51.616164 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8932e-99bb-4fdd-8fc1-06d0b9ef1bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:51.616954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8932e-99bb-4fdd-8fc1-06d0b9ef1bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:51.617239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8932e-99bb-4fdd-8fc1-06d0b9ef1bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:51.617516 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8932e-99bb-4fdd-8fc1-06d0b9ef1bb9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:51.629996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8932e-99bb-4fdd-8fc1-06d0b9ef1bb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:51.635096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-a2d8932e-99bb-4fdd-8fc1-06d0b9ef1bb9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:07:51.635096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:51.635096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:51.635096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:51.635096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:51.635096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:51.635096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:51.635096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:51.635096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:51.635096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:51.635096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:51.635096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:51.635096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:51.635096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:51.635096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:51.635096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:51.636824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:51.636824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:51.636824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:51.636824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:51.636824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:51.636824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:51.636824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:51.636824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:51.636824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:51.636824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:51.636824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:51.636824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:51.636824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:51.636824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:51.636824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:51.636824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:51.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:51.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:51.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:51.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:51.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:51.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:51.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:51.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:51.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:51.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:51.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:07:51.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:07:51.638268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 90/185] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:51.639851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8116c9e-0d6a-47b1-886d-5852f8d746da None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:51.639851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8116c9e-0d6a-47b1-886d-5852f8d746da None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:51.643919 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8116c9e-0d6a-47b1-886d-5852f8d746da None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:51.645948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 91/186] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:51.647664 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8116c9e-0d6a-47b1-886d-5852f8d746da None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:51.681703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:51.742865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0877b9-6995-4936-9a99-279f62ebdcc1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:51.758554 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8116c9e-0d6a-47b1-886d-5852f8d746da None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:51.782810 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b3513-4e01-4856-945e-56a795bd9553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:51.783634 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b3513-4e01-4856-945e-56a795bd9553 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:51.783919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b3513-4e01-4856-945e-56a795bd9553 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:51.784194 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b3513-4e01-4856-945e-56a795bd9553 None admin] PATH_INFO: `/v3/projects/f9c2c60f747b4390b69a797a1d5781f7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:51.785425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b3513-4e01-4856-945e-56a795bd9553 None admin] RBAC: Authorizing `identity:get_project(project_id=f9c2c60f747b4390b69a797a1d5781f7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:51.786869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b3513-4e01-4856-945e-56a795bd9553 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:51.788692 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 96/187] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:51 2022] GET /identity/v3/projects/f9c2c60f747b4390b69a797a1d5781f7 => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:51.795054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b3513-4e01-4856-945e-56a795bd9553 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:51.800291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa8b59-7a14-4dbf-aaf4-1c89b7043db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:51.800921 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa8b59-7a14-4dbf-aaf4-1c89b7043db9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:51.801200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa8b59-7a14-4dbf-aaf4-1c89b7043db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:51.801472 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa8b59-7a14-4dbf-aaf4-1c89b7043db9 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:51.802149 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa8b59-7a14-4dbf-aaf4-1c89b7043db9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:51.802451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa8b59-7a14-4dbf-aaf4-1c89b7043db9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:51.803894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa8b59-7a14-4dbf-aaf4-1c89b7043db9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:51.815327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 92/188] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:07:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:07:51.816676 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281c2ce-f8e0-48c6-b124-c7dc83c4327a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:51.816963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281c2ce-f8e0-48c6-b124-c7dc83c4327a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:51.817231 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281c2ce-f8e0-48c6-b124-c7dc83c4327a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:51.835388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-4281c2ce-f8e0-48c6-b124-c7dc83c4327a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:51.836605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4281c2ce-f8e0-48c6-b124-c7dc83c4327a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:51.893469 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:51.910768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 93/189] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:51.911740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281c2ce-f8e0-48c6-b124-c7dc83c4327a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:51.918078 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b3513-4e01-4856-945e-56a795bd9553 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:51.936550 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:51.955289 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79f914-7fc0-478c-ab8e-85fe03b9b500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:51.955967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79f914-7fc0-478c-ab8e-85fe03b9b500 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:51.956244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79f914-7fc0-478c-ab8e-85fe03b9b500 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:51.956514 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79f914-7fc0-478c-ab8e-85fe03b9b500 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:51.967122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79f914-7fc0-478c-ab8e-85fe03b9b500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:51.972351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-9e79f914-7fc0-478c-ab8e-85fe03b9b500 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:07:51.972351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:51.972351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:51.972351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:51.972351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:51.972351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:51.972351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:51.972351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:51.972351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:51.972351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:51.972351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:51.972351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:51.972351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:51.972351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:51.972351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:51.972351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:51.973244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:51.973244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:51.973244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:51.973244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:51.973244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:51.973244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:51.973244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:51.973244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:51.973244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:51.973244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:51.973244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:51.973244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:51.973244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:51.973244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:51.973244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:51.973244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:51.974090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:51.974090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:51.974090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:51.974090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:51.974090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:51.974090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:51.974090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:51.974090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:51.974090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:51.974090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:51.974090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:07:51.974090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:07:51.974090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 97/190] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:07:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:51.974762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79f914-7fc0-478c-ab8e-85fe03b9b500 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:52.023854 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281c2ce-f8e0-48c6-b124-c7dc83c4327a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:52.061762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592417d3-5113-4a40-b2a0-49c1581c2fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:52.062097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592417d3-5113-4a40-b2a0-49c1581c2fea None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:52.062371 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592417d3-5113-4a40-b2a0-49c1581c2fea None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:52.062637 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592417d3-5113-4a40-b2a0-49c1581c2fea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:52.073334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592417d3-5113-4a40-b2a0-49c1581c2fea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:52.078136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-592417d3-5113-4a40-b2a0-49c1581c2fea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:07:52.078136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:52.078136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:52.078136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:52.078136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:52.078136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:52.078136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:52.078136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:52.078136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:52.078136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:52.078136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:52.078136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:52.078136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:52.078136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:52.078136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:52.078136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:52.079028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:52.079028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:52.079028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:52.079028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:52.079028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:52.079028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:52.079028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:52.079028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:52.079028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:52.079028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:52.079028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:52.079028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:52.079028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:52.079028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:52.079028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:52.079028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:52.079896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:52.079896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:52.079896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:52.079896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:52.079896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:52.079896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:52.079896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:52.079896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:52.079896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:52.079896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:52.079896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:07:52.079896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:07:52.079896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 94/191] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:07:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:52.080569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592417d3-5113-4a40-b2a0-49c1581c2fea None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:52.085639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79f914-7fc0-478c-ab8e-85fe03b9b500 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:52.121565 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5da33-40df-4201-a895-57ed224480f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:52.122166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5da33-40df-4201-a895-57ed224480f0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:52.122446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5da33-40df-4201-a895-57ed224480f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:52.122718 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5da33-40df-4201-a895-57ed224480f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:52.123435 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5da33-40df-4201-a895-57ed224480f0 None admin] RBAC: Adding query filter params (role.id=bbea734b8a704d7ca8ec61bd45536cbe, scope.project.id=f9c2c60f747b4390b69a797a1d5781f7, user.id=02b6e37825904c729c9cf6fd8e1f7dbd) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:52.123778 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5da33-40df-4201-a895-57ed224480f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:52.125235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5da33-40df-4201-a895-57ed224480f0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:52.130985 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 98/192] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:07:51 2022] GET /identity/v3/role_assignments?user.id=02b6e37825904c729c9cf6fd8e1f7dbd&scope.project.id=f9c2c60f747b4390b69a797a1d5781f7&role.id=bbea734b8a704d7ca8ec61bd45536cbe => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:52.131625 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5da33-40df-4201-a895-57ed224480f0 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:52.138261 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:52.189956 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:52.211037 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592417d3-5113-4a40-b2a0-49c1581c2fea None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:52.263180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e23d6-3930-40ea-bed4-f9c94c4cfe96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:52.263632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5da33-40df-4201-a895-57ed224480f0 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:52.263867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e23d6-3930-40ea-bed4-f9c94c4cfe96 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:52.264161 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e23d6-3930-40ea-bed4-f9c94c4cfe96 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:52.264407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e23d6-3930-40ea-bed4-f9c94c4cfe96 None admin] PATH_INFO: `/v3/users/1cbd4e62c7a0428ba026ac3c642204c9` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:52.265970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e23d6-3930-40ea-bed4-f9c94c4cfe96 None admin] RBAC: Authorizing `identity:get_user(user_id=1cbd4e62c7a0428ba026ac3c642204c9)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:52.267480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e23d6-3930-40ea-bed4-f9c94c4cfe96 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:52.269469 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 95/193] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:52 2022] GET /identity/v3/users/1cbd4e62c7a0428ba026ac3c642204c9 => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:52.270191 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e23d6-3930-40ea-bed4-f9c94c4cfe96 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:52.304211 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe1c06-64aa-4cde-803e-a8f1f575596d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:52.304737 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe1c06-64aa-4cde-803e-a8f1f575596d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:52.305020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe1c06-64aa-4cde-803e-a8f1f575596d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:52.305291 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe1c06-64aa-4cde-803e-a8f1f575596d None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:52.305966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe1c06-64aa-4cde-803e-a8f1f575596d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:52.306270 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe1c06-64aa-4cde-803e-a8f1f575596d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:52.308920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe1c06-64aa-4cde-803e-a8f1f575596d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:52.319094 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 99/194] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:07:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:52.320843 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe1c06-64aa-4cde-803e-a8f1f575596d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:52.394193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:52.400889 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e23d6-3930-40ea-bed4-f9c94c4cfe96 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:52.433865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe1c06-64aa-4cde-803e-a8f1f575596d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:52.435973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b2014-9a0c-4484-b6d1-4cc3f76f8bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:52.436579 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b2014-9a0c-4484-b6d1-4cc3f76f8bdb None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:52.436860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b2014-9a0c-4484-b6d1-4cc3f76f8bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:52.437133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b2014-9a0c-4484-b6d1-4cc3f76f8bdb None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:52.437794 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b2014-9a0c-4484-b6d1-4cc3f76f8bdb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:52.438312 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b2014-9a0c-4484-b6d1-4cc3f76f8bdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:52.439461 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b2014-9a0c-4484-b6d1-4cc3f76f8bdb None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:52.447631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 96/195] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:52.448305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b2014-9a0c-4484-b6d1-4cc3f76f8bdb None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:52.463437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:52.467715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35decad8-a643-4456-a2ea-501f9356dac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:52.468350 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35decad8-a643-4456-a2ea-501f9356dac7 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:52.468633 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35decad8-a643-4456-a2ea-501f9356dac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:52.468908 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35decad8-a643-4456-a2ea-501f9356dac7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:52.470017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35decad8-a643-4456-a2ea-501f9356dac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:52.471463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35decad8-a643-4456-a2ea-501f9356dac7 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:52.473053 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 100/196] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:07:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:52.476371 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35decad8-a643-4456-a2ea-501f9356dac7 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:52.547001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b2014-9a0c-4484-b6d1-4cc3f76f8bdb None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:52.581824 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35decad8-a643-4456-a2ea-501f9356dac7 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:52.582344 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ceab4-35b5-414d-9d6a-8c50d5c507a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:52.582684 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ceab4-35b5-414d-9d6a-8c50d5c507a0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:52.582965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ceab4-35b5-414d-9d6a-8c50d5c507a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:52.583241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ceab4-35b5-414d-9d6a-8c50d5c507a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:52.610837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:52.616460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab6473-9874-424c-b2d2-5b9e7c0ab8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:52.617087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab6473-9874-424c-b2d2-5b9e7c0ab8ef None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:52.617367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab6473-9874-424c-b2d2-5b9e7c0ab8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:52.617647 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab6473-9874-424c-b2d2-5b9e7c0ab8ef None admin] PATH_INFO: `/v3/users/45a6bfb5faab4ef8aa91d24075adb297` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:52.630582 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab6473-9874-424c-b2d2-5b9e7c0ab8ef None admin] RBAC: Authorizing `identity:get_user(user_id=45a6bfb5faab4ef8aa91d24075adb297)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:52.632094 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab6473-9874-424c-b2d2-5b9e7c0ab8ef None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:52.633706 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 101/197] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:52 2022] GET /identity/v3/users/45a6bfb5faab4ef8aa91d24075adb297 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:52.639002 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab6473-9874-424c-b2d2-5b9e7c0ab8ef None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:52.650972 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ceab4-35b5-414d-9d6a-8c50d5c507a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:52.652489 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ceab4-35b5-414d-9d6a-8c50d5c507a0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:52.667332 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:52.732816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab6473-9874-424c-b2d2-5b9e7c0ab8ef None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:52.750156 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 97/198] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:07:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:07:52.750943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ceab4-35b5-414d-9d6a-8c50d5c507a0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:52.766481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f30b7-e9e8-4c50-820d-b97629837be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:52.767110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f30b7-e9e8-4c50-820d-b97629837be2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:52.767392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f30b7-e9e8-4c50-820d-b97629837be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:52.767694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f30b7-e9e8-4c50-820d-b97629837be2 None admin] PATH_INFO: `/v3/users/45a6bfb5faab4ef8aa91d24075adb297` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:52.768816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4f30b7-e9e8-4c50-820d-b97629837be2 None admin] RBAC: Authorizing `identity:get_user(user_id=45a6bfb5faab4ef8aa91d24075adb297)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:52.770258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4f30b7-e9e8-4c50-820d-b97629837be2 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:52.771853 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 102/199] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:52 2022] GET /identity/v3/users/45a6bfb5faab4ef8aa91d24075adb297 => generated 283 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:52.772600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f30b7-e9e8-4c50-820d-b97629837be2 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:52.828773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:52.851012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ceab4-35b5-414d-9d6a-8c50d5c507a0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:52.864575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:52.869252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f30b7-e9e8-4c50-820d-b97629837be2 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:52.884720 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7217717-bbc7-4935-b625-827f61bc0ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:52.885387 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7217717-bbc7-4935-b625-827f61bc0ec0 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:52.885672 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7217717-bbc7-4935-b625-827f61bc0ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:52.885949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7217717-bbc7-4935-b625-827f61bc0ec0 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:52.886601 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7217717-bbc7-4935-b625-827f61bc0ec0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:52.888096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7217717-bbc7-4935-b625-827f61bc0ec0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:52.902962 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32db93-1b18-4924-b073-b855be638fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:52.903596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32db93-1b18-4924-b073-b855be638fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:52.903897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32db93-1b18-4924-b073-b855be638fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:52.904211 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32db93-1b18-4924-b073-b855be638fb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:52.909198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 98/200] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:07:52 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:07:52.912835 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7217717-bbc7-4935-b625-827f61bc0ec0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:52.981426 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32db93-1b18-4924-b073-b855be638fb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:52.983232 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32db93-1b18-4924-b073-b855be638fb7 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:53.043484 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7217717-bbc7-4935-b625-827f61bc0ec0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:53.046510 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:53.088975 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea73e6-83e0-461b-9523-16abd6005e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:53.089900 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea73e6-83e0-461b-9523-16abd6005e9f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:53.090183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea73e6-83e0-461b-9523-16abd6005e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:53.090464 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea73e6-83e0-461b-9523-16abd6005e9f None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:53.091684 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea73e6-83e0-461b-9523-16abd6005e9f None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:53.093303 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea73e6-83e0-461b-9523-16abd6005e9f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:53.095295 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 99/201] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:52 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:53.096096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea73e6-83e0-461b-9523-16abd6005e9f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:53.098136 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 103/202] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:07:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:07:53.101280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32db93-1b18-4924-b073-b855be638fb7 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:53.143613 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:53.218140 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea73e6-83e0-461b-9523-16abd6005e9f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:53.228572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32db93-1b18-4924-b073-b855be638fb7 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:53.265042 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9d601-b314-4ce2-9379-4b434883c943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:53.265259 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9d601-b314-4ce2-9379-4b434883c943 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:53.265542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9d601-b314-4ce2-9379-4b434883c943 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:53.265805 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9d601-b314-4ce2-9379-4b434883c943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:53.271769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaff4ae-b37f-4963-9ce9-1422f917b764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:53.272673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaff4ae-b37f-4963-9ce9-1422f917b764 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:53.273180 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaff4ae-b37f-4963-9ce9-1422f917b764 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:53.273651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaff4ae-b37f-4963-9ce9-1422f917b764 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:53.274679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaff4ae-b37f-4963-9ce9-1422f917b764 None admin] RBAC: Adding query filter params (role.id=061fff977e1f4d13a7b5e8e70edc3e92, scope.project.id=b427802e13664086974c8d53f5bfbc88, user.id=1cbd4e62c7a0428ba026ac3c642204c9) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:53.275207 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaff4ae-b37f-4963-9ce9-1422f917b764 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:53.277041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaff4ae-b37f-4963-9ce9-1422f917b764 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:53.284400 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 104/203] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:07:53 2022] GET /identity/v3/role_assignments?user.id=1cbd4e62c7a0428ba026ac3c642204c9&scope.project.id=b427802e13664086974c8d53f5bfbc88&role.id=061fff977e1f4d13a7b5e8e70edc3e92 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:07:53.285263 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaff4ae-b37f-4963-9ce9-1422f917b764 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:53.316752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:53.359021 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9d601-b314-4ce2-9379-4b434883c943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:53.360563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9d601-b314-4ce2-9379-4b434883c943 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:53.413752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaff4ae-b37f-4963-9ce9-1422f917b764 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:53.454325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146496d7-56c6-4e47-9018-baa3e5822903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:53.455350 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146496d7-56c6-4e47-9018-baa3e5822903 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:53.455998 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146496d7-56c6-4e47-9018-baa3e5822903 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:53.456474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146496d7-56c6-4e47-9018-baa3e5822903 None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:53.457912 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146496d7-56c6-4e47-9018-baa3e5822903 None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:53.459669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146496d7-56c6-4e47-9018-baa3e5822903 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:53.462157 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 105/204] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:53 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:53.469573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146496d7-56c6-4e47-9018-baa3e5822903 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:53.474058 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:53.554509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 100/205] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:07:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:07:53.560655 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9d601-b314-4ce2-9379-4b434883c943 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:53.592044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:53.603775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146496d7-56c6-4e47-9018-baa3e5822903 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:53.642450 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612f422-139a-4ba7-b999-2a4a9c021501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:53.643292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612f422-139a-4ba7-b999-2a4a9c021501 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:53.643753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612f422-139a-4ba7-b999-2a4a9c021501 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:53.644175 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612f422-139a-4ba7-b999-2a4a9c021501 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:53.645080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612f422-139a-4ba7-b999-2a4a9c021501 None admin] RBAC: Adding query filter params (role.id=2c11a95c1ecf4988a95aad8d96ea56e9, scope.project.id=d844147ccee64e23b49af3bf86ac4c6f, user.id=45a6bfb5faab4ef8aa91d24075adb297) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:53.645625 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612f422-139a-4ba7-b999-2a4a9c021501 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:53.647348 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612f422-139a-4ba7-b999-2a4a9c021501 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:53.653922 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 106/206] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:07:53 2022] GET /identity/v3/role_assignments?user.id=45a6bfb5faab4ef8aa91d24075adb297&scope.project.id=d844147ccee64e23b49af3bf86ac4c6f&role.id=2c11a95c1ecf4988a95aad8d96ea56e9 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:53.680532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9d601-b314-4ce2-9379-4b434883c943 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:53.720845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34451fe-776a-4179-80e0-d4c068f88ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:53.721496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34451fe-776a-4179-80e0-d4c068f88ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:53.721825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34451fe-776a-4179-80e0-d4c068f88ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:53.722105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34451fe-776a-4179-80e0-d4c068f88ce7 None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:53.723311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34451fe-776a-4179-80e0-d4c068f88ce7 None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:53.724932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34451fe-776a-4179-80e0-d4c068f88ce7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:53.726692 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 101/207] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:53 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:53.733181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612f422-139a-4ba7-b999-2a4a9c021501 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:53.852642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612f422-139a-4ba7-b999-2a4a9c021501 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:53.889089 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e743f-caf2-49fb-b40f-babb38b4032f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:53.889708 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e743f-caf2-49fb-b40f-babb38b4032f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:53.889988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e743f-caf2-49fb-b40f-babb38b4032f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:53.890319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e743f-caf2-49fb-b40f-babb38b4032f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:53.891106 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e743f-caf2-49fb-b40f-babb38b4032f None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=d844147ccee64e23b49af3bf86ac4c6f, user.id=45a6bfb5faab4ef8aa91d24075adb297) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:53.891416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e743f-caf2-49fb-b40f-babb38b4032f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:53.892944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e743f-caf2-49fb-b40f-babb38b4032f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:53.899058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 107/208] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:07:53 2022] GET /identity/v3/role_assignments?user.id=45a6bfb5faab4ef8aa91d24075adb297&scope.project.id=d844147ccee64e23b49af3bf86ac4c6f&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:55.329184 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6cd98-13ba-4ca6-9241-85a5746b88c5 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:55.329831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6cd98-13ba-4ca6-9241-85a5746b88c5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:55.329831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6cd98-13ba-4ca6-9241-85a5746b88c5 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:55.331302 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 102/209] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:55.339537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8be9f7-89c2-4517-ae81-f6e7ba810794 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:55.339537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8be9f7-89c2-4517-ae81-f6e7ba810794 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:55.339537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8be9f7-89c2-4517-ae81-f6e7ba810794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:55.357093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-3d8be9f7-89c2-4517-ae81-f6e7ba810794 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:55.358553 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3d8be9f7-89c2-4517-ae81-f6e7ba810794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:55.366803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fd993-f71c-4bf5-a937-fd358ced82b7 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:55.367086 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fd993-f71c-4bf5-a937-fd358ced82b7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:55.367357 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fd993-f71c-4bf5-a937-fd358ced82b7 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:55.368720 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 103/210] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:55.373628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8329d-a4ae-43fb-80e7-b1e806f5eeae None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:55.373744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:55.373915 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8329d-a4ae-43fb-80e7-b1e806f5eeae None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:55.374178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8329d-a4ae-43fb-80e7-b1e806f5eeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:55.387960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:55.393513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ccd8329d-a4ae-43fb-80e7-b1e806f5eeae None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:55.394716 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ccd8329d-a4ae-43fb-80e7-b1e806f5eeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:55.431622 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 108/211] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:55.470748 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 104/212] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:55.705539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8be9f7-89c2-4517-ae81-f6e7ba810794 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:55.741246 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8329d-a4ae-43fb-80e7-b1e806f5eeae None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:55.834261 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8be9f7-89c2-4517-ae81-f6e7ba810794 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:55.857425 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8329d-a4ae-43fb-80e7-b1e806f5eeae None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:55.870382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0b7af-f192-4865-b181-1e98d7feccab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:55.871040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0b7af-f192-4865-b181-1e98d7feccab None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:55.871563 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0b7af-f192-4865-b181-1e98d7feccab None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:55.871736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0b7af-f192-4865-b181-1e98d7feccab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:55.882638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0b7af-f192-4865-b181-1e98d7feccab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:55.887848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-d0b0b7af-f192-4865-b181-1e98d7feccab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:07:55.887848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:55.887848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:55.887848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:55.887848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:55.887848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:55.887848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:55.887848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:55.887848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:55.887848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:55.887848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:55.887848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:55.887848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:55.887848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:55.887848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:55.887848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:55.888752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:55.888752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:55.888752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:55.888752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:55.888752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:55.888752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:55.888752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:55.888752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:55.888752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:55.888752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:55.888752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:55.888752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:55.888752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:55.888752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:55.888752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:55.888752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:55.889549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:55.889549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:55.889549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:55.889549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:55.889549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:55.889549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:55.889549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:55.889549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:55.889549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:55.889549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:55.889549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:07:55.889549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:07:55.889549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 109/213] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:55.896743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe6adc-cad4-47bb-b902-2b15bafb8447 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:55.897037 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe6adc-cad4-47bb-b902-2b15bafb8447 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:55.897321 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe6adc-cad4-47bb-b902-2b15bafb8447 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:55.902803 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 110/214] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:55.902803 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe6adc-cad4-47bb-b902-2b15bafb8447 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:55.907329 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0b7a6-a87b-4a8f-a55f-791f8bb065de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:55.907996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0b7a6-a87b-4a8f-a55f-791f8bb065de None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:55.908275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0b7a6-a87b-4a8f-a55f-791f8bb065de None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:55.908542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0b7a6-a87b-4a8f-a55f-791f8bb065de None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:55.909631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0b7a6-a87b-4a8f-a55f-791f8bb065de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:55.911074 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0b7a6-a87b-4a8f-a55f-791f8bb065de None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:55.911826 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:55.915551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 105/215] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:07:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:07:55.915551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832ebf9-2ccc-41da-af4a-22a34997f494 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:55.915551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832ebf9-2ccc-41da-af4a-22a34997f494 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:55.915551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832ebf9-2ccc-41da-af4a-22a34997f494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:55.922862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:55.935878 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-9832ebf9-2ccc-41da-af4a-22a34997f494 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:55.937280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9832ebf9-2ccc-41da-af4a-22a34997f494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:56.015634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 106/216] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:56.015634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832ebf9-2ccc-41da-af4a-22a34997f494 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:56.021352 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe6adc-cad4-47bb-b902-2b15bafb8447 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:56.058734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b06f71-b5f1-40d5-8926-4897674b896b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:56.059196 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b06f71-b5f1-40d5-8926-4897674b896b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:56.059561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b06f71-b5f1-40d5-8926-4897674b896b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:56.059919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b06f71-b5f1-40d5-8926-4897674b896b None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:56.060640 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b06f71-b5f1-40d5-8926-4897674b896b None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:56.061007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b06f71-b5f1-40d5-8926-4897674b896b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:56.062515 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b06f71-b5f1-40d5-8926-4897674b896b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:56.076426 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 111/217] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:07:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:56.077827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b06f71-b5f1-40d5-8926-4897674b896b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:56.141706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832ebf9-2ccc-41da-af4a-22a34997f494 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:56.149587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:56.178599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6963b-ee11-4cfd-b4b6-0baffcbdbe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:56.179212 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6963b-ee11-4cfd-b4b6-0baffcbdbe74 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:56.179490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6963b-ee11-4cfd-b4b6-0baffcbdbe74 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:56.179787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6963b-ee11-4cfd-b4b6-0baffcbdbe74 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:56.180434 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6963b-ee11-4cfd-b4b6-0baffcbdbe74 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:56.181771 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b06f71-b5f1-40d5-8926-4897674b896b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:56.181871 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6963b-ee11-4cfd-b4b6-0baffcbdbe74 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:56.190053 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:56.206515 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 107/218] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:07:56 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:07:56.207736 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f6993-03fc-43c4-9680-9647371dd4d5 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:56.208031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f6993-03fc-43c4-9680-9647371dd4d5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:56.208522 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f6993-03fc-43c4-9680-9647371dd4d5 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:56.209615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 108/219] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:56.215346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bb6ff-9226-409f-ae6f-fc6642973877 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:56.215694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bb6ff-9226-409f-ae6f-fc6642973877 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:56.216013 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bb6ff-9226-409f-ae6f-fc6642973877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:56.245712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-489bb6ff-9226-409f-ae6f-fc6642973877 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:56.246911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-489bb6ff-9226-409f-ae6f-fc6642973877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:56.247407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398394f-67e0-45d0-a7b9-ab42c396cad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:56.248506 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398394f-67e0-45d0-a7b9-ab42c396cad1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:56.248995 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398394f-67e0-45d0-a7b9-ab42c396cad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:56.249530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398394f-67e0-45d0-a7b9-ab42c396cad1 None admin] PATH_INFO: `/v3/users/f46600603b394108b26ea29d87152d22` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:56.271169 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398394f-67e0-45d0-a7b9-ab42c396cad1 None admin] RBAC: Authorizing `identity:get_user(user_id=f46600603b394108b26ea29d87152d22)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:56.273333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398394f-67e0-45d0-a7b9-ab42c396cad1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:56.276608 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 112/220] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:56 2022] GET /identity/v3/users/f46600603b394108b26ea29d87152d22 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:56.287561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398394f-67e0-45d0-a7b9-ab42c396cad1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:56.337747 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 109/221] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:07:56.339372 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bb6ff-9226-409f-ae6f-fc6642973877 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:56.423606 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398394f-67e0-45d0-a7b9-ab42c396cad1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:56.457822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:56.468881 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d1f33-54dd-4529-b126-ea706a16f614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:56.469494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d1f33-54dd-4529-b126-ea706a16f614 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:56.469771 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d1f33-54dd-4529-b126-ea706a16f614 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:56.470037 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d1f33-54dd-4529-b126-ea706a16f614 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:56.482804 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bb6ff-9226-409f-ae6f-fc6642973877 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:56.516480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:56.532080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea970a-c527-4722-a9c2-fb545f6b4add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:56.532657 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea970a-c527-4722-a9c2-fb545f6b4add None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:56.532941 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea970a-c527-4722-a9c2-fb545f6b4add None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:56.533267 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea970a-c527-4722-a9c2-fb545f6b4add None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:56.550247 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea970a-c527-4722-a9c2-fb545f6b4add None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:56.556518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-03ea970a-c527-4722-a9c2-fb545f6b4add None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:07:56.556518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:56.556518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:56.556518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:56.556518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:56.556518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:56.556518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:56.556518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:56.556518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:56.556518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:56.556518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:56.556518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:56.556518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:56.556518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:56.556518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:56.556518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:56.557474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:56.557474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:56.557474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:56.557474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:56.557474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:56.557474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:56.557474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:56.557474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:56.557474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:56.557474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:56.557474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:56.557474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:56.557474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:56.557474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:56.557474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:56.557474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:56.558285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:56.558285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:56.558285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:56.558285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:56.558285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:56.558285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:56.558285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:56.558285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:56.558285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:56.558285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:56.558285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:07:56.558285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:07:56.560423 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d1f33-54dd-4529-b126-ea706a16f614 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:56.560711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 110/222] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:07:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:56.563568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d1f33-54dd-4529-b126-ea706a16f614 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:56.567115 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea970a-c527-4722-a9c2-fb545f6b4add None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:56.692780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 113/223] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:07:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:07:56.693639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d1f33-54dd-4529-b126-ea706a16f614 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:56.706564 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea970a-c527-4722-a9c2-fb545f6b4add None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:56.726906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:56.750323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0e2c2-322a-48e4-8130-84acabe7c8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:56.751160 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0e2c2-322a-48e4-8130-84acabe7c8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:56.751642 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0e2c2-322a-48e4-8130-84acabe7c8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:56.752067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0e2c2-322a-48e4-8130-84acabe7c8f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:56.752871 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0e2c2-322a-48e4-8130-84acabe7c8f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:56.753321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0e2c2-322a-48e4-8130-84acabe7c8f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:56.754836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0e2c2-322a-48e4-8130-84acabe7c8f8 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:56.765590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 111/224] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:07:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:56.766608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0e2c2-322a-48e4-8130-84acabe7c8f8 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:56.804017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d1f33-54dd-4529-b126-ea706a16f614 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:56.817907 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:56.840997 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ce7ae-2fcf-4494-b58b-92363cf649f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:56.841726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ce7ae-2fcf-4494-b58b-92363cf649f7 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:56.842071 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ce7ae-2fcf-4494-b58b-92363cf649f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:56.842465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ce7ae-2fcf-4494-b58b-92363cf649f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:56.853468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ce7ae-2fcf-4494-b58b-92363cf649f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:56.859283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-001ce7ae-2fcf-4494-b58b-92363cf649f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:07:56.859283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:56.859283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:56.859283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:56.859283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:56.859283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:56.859283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:56.859283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:56.859283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:56.859283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:56.859283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:56.859283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:56.859283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:56.859283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:56.859283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:56.859283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:56.860226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:56.860226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:56.860226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:56.860226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:56.860226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:56.860226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:56.860226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:56.860226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:56.860226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:56.860226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:56.860226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:56.860226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:56.860226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:56.860226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:56.860226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:56.860226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:56.861041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:56.861041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:56.861041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:56.861041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:56.861041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:56.861041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:56.861041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:56.861041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:56.861041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:56.861041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:56.861041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:07:56.861041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:07:56.861041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 114/225] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:07:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:56.861709 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ce7ae-2fcf-4494-b58b-92363cf649f7 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:56.889457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0e2c2-322a-48e4-8130-84acabe7c8f8 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:56.934643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8152feb-a7a1-474a-ac6a-67db11556abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:56.935999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152feb-a7a1-474a-ac6a-67db11556abc None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:56.936737 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152feb-a7a1-474a-ac6a-67db11556abc None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:56.937154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8152feb-a7a1-474a-ac6a-67db11556abc None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:56.938830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8152feb-a7a1-474a-ac6a-67db11556abc None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:56.940461 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8152feb-a7a1-474a-ac6a-67db11556abc None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:56.942547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 112/226] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:56 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:56.944132 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8152feb-a7a1-474a-ac6a-67db11556abc None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:56.968829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:56.985258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ce7ae-2fcf-4494-b58b-92363cf649f7 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:57.029541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43bc778-ff11-43c3-988e-a29267374afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:57.030216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43bc778-ff11-43c3-988e-a29267374afd None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:57.030501 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43bc778-ff11-43c3-988e-a29267374afd None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:57.031043 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43bc778-ff11-43c3-988e-a29267374afd None admin] PATH_INFO: `/v3/users/45a6bfb5faab4ef8aa91d24075adb297` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:57.032607 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43bc778-ff11-43c3-988e-a29267374afd None admin] RBAC: Authorizing `identity:get_user(user_id=45a6bfb5faab4ef8aa91d24075adb297)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:57.034081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43bc778-ff11-43c3-988e-a29267374afd None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:57.036466 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 115/227] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:56 2022] GET /identity/v3/users/45a6bfb5faab4ef8aa91d24075adb297 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:57.039741 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43bc778-ff11-43c3-988e-a29267374afd None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:57.068423 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8152feb-a7a1-474a-ac6a-67db11556abc None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:57.082001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:57.111607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc330c9f-be00-4ff9-a6aa-233060e191d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:57.112467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc330c9f-be00-4ff9-a6aa-233060e191d7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:57.112845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc330c9f-be00-4ff9-a6aa-233060e191d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:57.113212 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc330c9f-be00-4ff9-a6aa-233060e191d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:57.113981 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc330c9f-be00-4ff9-a6aa-233060e191d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:57.114394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc330c9f-be00-4ff9-a6aa-233060e191d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:57.116065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc330c9f-be00-4ff9-a6aa-233060e191d7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:57.127167 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 113/228] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:57.128394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc330c9f-be00-4ff9-a6aa-233060e191d7 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:57.179829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43bc778-ff11-43c3-988e-a29267374afd None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:57.224927 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b12f3e-a488-4d43-8995-64c3df32dbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:57.226224 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b12f3e-a488-4d43-8995-64c3df32dbb6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:57.226655 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b12f3e-a488-4d43-8995-64c3df32dbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:57.227494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b12f3e-a488-4d43-8995-64c3df32dbb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:57.228692 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b12f3e-a488-4d43-8995-64c3df32dbb6 None admin] RBAC: Adding query filter params (role.id=bbea734b8a704d7ca8ec61bd45536cbe, scope.project.id=d844147ccee64e23b49af3bf86ac4c6f, user.id=f46600603b394108b26ea29d87152d22) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:57.229715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b12f3e-a488-4d43-8995-64c3df32dbb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:57.232118 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b12f3e-a488-4d43-8995-64c3df32dbb6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:57.240138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 116/229] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:07:57 2022] GET /identity/v3/role_assignments?user.id=f46600603b394108b26ea29d87152d22&scope.project.id=d844147ccee64e23b49af3bf86ac4c6f&role.id=bbea734b8a704d7ca8ec61bd45536cbe => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:57.244945 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b12f3e-a488-4d43-8995-64c3df32dbb6 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:57.264751 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc330c9f-be00-4ff9-a6aa-233060e191d7 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:57.274174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:57.340411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a653122-859f-4484-b74e-39a82caaead6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:57.341183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a653122-859f-4484-b74e-39a82caaead6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:57.341560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a653122-859f-4484-b74e-39a82caaead6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:57.341927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a653122-859f-4484-b74e-39a82caaead6 None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:57.344382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a653122-859f-4484-b74e-39a82caaead6 None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:57.346104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a653122-859f-4484-b74e-39a82caaead6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:57.348407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 114/230] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:57 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:57.369020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a653122-859f-4484-b74e-39a82caaead6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:57.378857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b12f3e-a488-4d43-8995-64c3df32dbb6 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:57.420102 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d6e06-d8f5-4208-81ac-2b06615e7d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:57.421304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d6e06-d8f5-4208-81ac-2b06615e7d0d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:57.421813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d6e06-d8f5-4208-81ac-2b06615e7d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:57.422275 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d6e06-d8f5-4208-81ac-2b06615e7d0d None admin] PATH_INFO: `/v3/users/45a6bfb5faab4ef8aa91d24075adb297` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:57.424619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d6e06-d8f5-4208-81ac-2b06615e7d0d None admin] RBAC: Authorizing `identity:get_user(user_id=45a6bfb5faab4ef8aa91d24075adb297)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:57.427323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d6e06-d8f5-4208-81ac-2b06615e7d0d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:57.430424 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 117/231] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:57 2022] GET /identity/v3/users/45a6bfb5faab4ef8aa91d24075adb297 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:57.437784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d6e06-d8f5-4208-81ac-2b06615e7d0d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:57.462154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:57.558847 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a653122-859f-4484-b74e-39a82caaead6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:57.584354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:57.594212 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd7fd9-d68d-4e59-a272-5d08d06c1fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:57.594807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd7fd9-d68d-4e59-a272-5d08d06c1fef None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:57.595086 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd7fd9-d68d-4e59-a272-5d08d06c1fef None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:57.595427 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd7fd9-d68d-4e59-a272-5d08d06c1fef None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f/users/45a6bfb5faab4ef8aa91d24075adb297/roles/2c11a95c1ecf4988a95aad8d96ea56e9` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:57.597582 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd7fd9-d68d-4e59-a272-5d08d06c1fef None admin] RBAC: Authorizing `identity:create_grant(project_id=d844147ccee64e23b49af3bf86ac4c6f, user_id=45a6bfb5faab4ef8aa91d24075adb297, role_id=2c11a95c1ecf4988a95aad8d96ea56e9)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:57.599120 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd7fd9-d68d-4e59-a272-5d08d06c1fef None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:57.601454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d6e06-d8f5-4208-81ac-2b06615e7d0d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:57.619532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 115/232] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:07:57 2022] PUT /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f/users/45a6bfb5faab4ef8aa91d24075adb297/roles/2c11a95c1ecf4988a95aad8d96ea56e9 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:07:57.670023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b0a54-e11c-427a-88a9-fe80113041b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:57.670671 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b0a54-e11c-427a-88a9-fe80113041b4 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:57.670953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b0a54-e11c-427a-88a9-fe80113041b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:57.671225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b0a54-e11c-427a-88a9-fe80113041b4 None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:57.676373 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b0a54-e11c-427a-88a9-fe80113041b4 None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:57.677873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b0a54-e11c-427a-88a9-fe80113041b4 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:57.679946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 118/233] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:07:57 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:57.685701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd7fd9-d68d-4e59-a272-5d08d06c1fef None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:57.839891 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:57.845524 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd7fd9-d68d-4e59-a272-5d08d06c1fef None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:57.886417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d205e-4853-4e52-a627-8c2ff63808a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:57.887114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d205e-4853-4e52-a627-8c2ff63808a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:57.887367 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d205e-4853-4e52-a627-8c2ff63808a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:57.887673 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d205e-4853-4e52-a627-8c2ff63808a4 None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f/users/45a6bfb5faab4ef8aa91d24075adb297/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:57.889850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d205e-4853-4e52-a627-8c2ff63808a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d844147ccee64e23b49af3bf86ac4c6f, user_id=45a6bfb5faab4ef8aa91d24075adb297, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:57.891393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d205e-4853-4e52-a627-8c2ff63808a4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:57.908506 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 116/234] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:07:57 2022] PUT /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f/users/45a6bfb5faab4ef8aa91d24075adb297/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:07:58.862707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb4c68-2664-460e-bbcc-418d9d4571ef None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:58.863798 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb4c68-2664-460e-bbcc-418d9d4571ef None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:58.864306 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb4c68-2664-460e-bbcc-418d9d4571ef None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:58.866296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 119/235] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:58.872883 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c52e2b-34b7-4a92-81dd-f84416c2e888 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:58.873838 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c52e2b-34b7-4a92-81dd-f84416c2e888 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:58.874349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c52e2b-34b7-4a92-81dd-f84416c2e888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:58.897177 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b8c52e2b-34b7-4a92-81dd-f84416c2e888 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:58.898700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b8c52e2b-34b7-4a92-81dd-f84416c2e888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:58.993756 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 117/236] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:59.017126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e0c9c-0af9-43dd-86ca-5dcb660b1a6f None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:59.017750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e0c9c-0af9-43dd-86ca-5dcb660b1a6f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:59.018229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e0c9c-0af9-43dd-86ca-5dcb660b1a6f None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:59.019791 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 120/237] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:07:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:07:59.025169 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a067d6-4893-420e-955b-24a1152eaeb2 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:59.025457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a067d6-4893-420e-955b-24a1152eaeb2 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:59.025763 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a067d6-4893-420e-955b-24a1152eaeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:59.052470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-86a067d6-4893-420e-955b-24a1152eaeb2 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:07:59.053671 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-86a067d6-4893-420e-955b-24a1152eaeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:07:59.153217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 118/238] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:07:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:07:59.379353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e0c9c-0af9-43dd-86ca-5dcb660b1a6f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:59.506939 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e0c9c-0af9-43dd-86ca-5dcb660b1a6f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:59.516971 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a067d6-4893-420e-955b-24a1152eaeb2 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:59.545690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:59.568406 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972876d-94e5-4ac7-8dc8-bf94478f5bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:59.569049 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972876d-94e5-4ac7-8dc8-bf94478f5bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:59.569329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972876d-94e5-4ac7-8dc8-bf94478f5bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:59.569663 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972876d-94e5-4ac7-8dc8-bf94478f5bf8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:59.589493 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972876d-94e5-4ac7-8dc8-bf94478f5bf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:07:59.597664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-3972876d-94e5-4ac7-8dc8-bf94478f5bf8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:07:59.597664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:07:59.597664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:07:59.597664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:07:59.597664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:07:59.597664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:07:59.597664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:07:59.597664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:07:59.597664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:07:59.597664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:07:59.597664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:07:59.597664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:07:59.597664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:07:59.597664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:07:59.597664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:07:59.597664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:07:59.598621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:07:59.598621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:07:59.598621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:07:59.598621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:07:59.598621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:07:59.598621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:07:59.598621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:07:59.598621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:07:59.598621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:07:59.598621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:07:59.598621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:07:59.598621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:07:59.598621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:07:59.598621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:07:59.598621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:07:59.598621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:07:59.599427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:07:59.599427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:07:59.599427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:07:59.599427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:07:59.599427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:07:59.599427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:07:59.599427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:07:59.599427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:07:59.599427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:07:59.599427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:07:59.599427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:07:59.599427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:07:59.601659 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 121/239] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:07:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:07:59.606891 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972876d-94e5-4ac7-8dc8-bf94478f5bf8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:59.632779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:07:59.710859 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a067d6-4893-420e-955b-24a1152eaeb2 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:59.778393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972876d-94e5-4ac7-8dc8-bf94478f5bf8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:07:59.780397 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cd05c-7625-4099-a3c1-50faffb54b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:59.781297 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cd05c-7625-4099-a3c1-50faffb54b78 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:59.781575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cd05c-7625-4099-a3c1-50faffb54b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:59.781843 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cd05c-7625-4099-a3c1-50faffb54b78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:59.784442 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cd05c-7625-4099-a3c1-50faffb54b78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:59.785904 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cd05c-7625-4099-a3c1-50faffb54b78 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:59.787521 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 119/240] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:07:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:59.811834 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cd05c-7625-4099-a3c1-50faffb54b78 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:59.848975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9972f2-4019-4ea2-a0b5-d47ba8f7ce1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:07:59.849590 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9972f2-4019-4ea2-a0b5-d47ba8f7ce1b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:07:59.849869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9972f2-4019-4ea2-a0b5-d47ba8f7ce1b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:07:59.850139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9972f2-4019-4ea2-a0b5-d47ba8f7ce1b None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:07:59.850803 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9972f2-4019-4ea2-a0b5-d47ba8f7ce1b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:07:59.851103 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9972f2-4019-4ea2-a0b5-d47ba8f7ce1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:07:59.852521 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9972f2-4019-4ea2-a0b5-d47ba8f7ce1b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:07:59.870350 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 122/241] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:07:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:07:59.876010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9972f2-4019-4ea2-a0b5-d47ba8f7ce1b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:07:59.983907 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:00.005200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cd05c-7625-4099-a3c1-50faffb54b78 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:00.031968 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:00.073938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9972f2-4019-4ea2-a0b5-d47ba8f7ce1b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:00.086126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24954f-b38e-48a0-bef6-1fd548d0528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:00.086755 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24954f-b38e-48a0-bef6-1fd548d0528f None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:00.087034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24954f-b38e-48a0-bef6-1fd548d0528f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:00.087305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24954f-b38e-48a0-bef6-1fd548d0528f None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:00.087990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24954f-b38e-48a0-bef6-1fd548d0528f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:00.089516 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24954f-b38e-48a0-bef6-1fd548d0528f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:00.128506 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 120/242] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:07:59 2022] POST /identity/v3/users => generated 293 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:08:00.137143 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e7c57-6f69-4796-883b-7b1e618ac183 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:00.137941 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e7c57-6f69-4796-883b-7b1e618ac183 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:00.138448 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e7c57-6f69-4796-883b-7b1e618ac183 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:00.140140 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 121/243] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:00.146829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b37d1-ddff-4031-9eb1-b8e0830c5fde None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:00.147454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b37d1-ddff-4031-9eb1-b8e0830c5fde None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:00.148381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b37d1-ddff-4031-9eb1-b8e0830c5fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:00.154875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca5164-326d-4642-a44b-b54a850948b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:00.156842 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca5164-326d-4642-a44b-b54a850948b2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:00.158284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca5164-326d-4642-a44b-b54a850948b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:00.158821 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca5164-326d-4642-a44b-b54a850948b2 None admin] PATH_INFO: `/v3/users/ba2fc87b5e1a4d76976a702bdb6397e2` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:00.183797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e45b37d1-ddff-4031-9eb1-b8e0830c5fde None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:00.184284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca5164-326d-4642-a44b-b54a850948b2 None admin] RBAC: Authorizing `identity:get_user(user_id=ba2fc87b5e1a4d76976a702bdb6397e2)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:00.185042 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e45b37d1-ddff-4031-9eb1-b8e0830c5fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:00.186287 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca5164-326d-4642-a44b-b54a850948b2 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:00.188344 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 123/244] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:07:59 2022] GET /identity/v3/users/ba2fc87b5e1a4d76976a702bdb6397e2 => generated 297 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:00.189794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72536190-2a18-4daf-81b9-a289e2e0b2e1 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:00.190406 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72536190-2a18-4daf-81b9-a289e2e0b2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:00.190921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72536190-2a18-4daf-81b9-a289e2e0b2e1 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:00.192387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 124/245] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:00.194485 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72536190-2a18-4daf-81b9-a289e2e0b2e1 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:00.278877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 122/246] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:00.280356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037f33e-d3b5-44eb-8d86-831a966d1fee None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:00.280976 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037f33e-d3b5-44eb-8d86-831a966d1fee None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:00.281507 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037f33e-d3b5-44eb-8d86-831a966d1fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:00.301299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e037f33e-d3b5-44eb-8d86-831a966d1fee None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:00.302339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e037f33e-d3b5-44eb-8d86-831a966d1fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:00.337152 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72536190-2a18-4daf-81b9-a289e2e0b2e1 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:00.376822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65246cce-2dc1-49de-b86b-ffc88a49faf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:00.377448 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65246cce-2dc1-49de-b86b-ffc88a49faf1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:00.377741 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65246cce-2dc1-49de-b86b-ffc88a49faf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:00.378075 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65246cce-2dc1-49de-b86b-ffc88a49faf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:00.382701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:00.398664 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 123/247] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:00.460648 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65246cce-2dc1-49de-b86b-ffc88a49faf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:00.462481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65246cce-2dc1-49de-b86b-ffc88a49faf1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:00.515537 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb5f0b-2ba7-441d-bb68-c29594a0381d None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:00.516152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb5f0b-2ba7-441d-bb68-c29594a0381d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:00.516727 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb5f0b-2ba7-441d-bb68-c29594a0381d None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:00.518984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 124/248] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:00.526598 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6e922-58c1-4247-b8e1-ba4447264279 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:00.527454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6e922-58c1-4247-b8e1-ba4447264279 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:00.527833 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6e922-58c1-4247-b8e1-ba4447264279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:00.539473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:00.554060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-71b6e922-58c1-4247-b8e1-ba4447264279 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:00.555186 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-71b6e922-58c1-4247-b8e1-ba4447264279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:00.593196 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 125/249] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 12:08:00.598315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65246cce-2dc1-49de-b86b-ffc88a49faf1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:00.648117 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:00.648833 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 125/250] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:00.649749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6e922-58c1-4247-b8e1-ba4447264279 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:00.746643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65246cce-2dc1-49de-b86b-ffc88a49faf1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:00.776338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6e922-58c1-4247-b8e1-ba4447264279 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:00.791934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18519c83-3844-40aa-be40-55ba2879e98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:00.792578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18519c83-3844-40aa-be40-55ba2879e98a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:00.792858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18519c83-3844-40aa-be40-55ba2879e98a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:00.793136 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18519c83-3844-40aa-be40-55ba2879e98a None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:00.794266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18519c83-3844-40aa-be40-55ba2879e98a None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:00.795810 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18519c83-3844-40aa-be40-55ba2879e98a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:00.797761 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 126/251] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:00 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:00.798541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18519c83-3844-40aa-be40-55ba2879e98a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:00.816281 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6660b8-0ffe-4e03-8bb3-803e7715e597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:00.816846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6660b8-0ffe-4e03-8bb3-803e7715e597 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:00.817137 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6660b8-0ffe-4e03-8bb3-803e7715e597 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:00.817446 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6660b8-0ffe-4e03-8bb3-803e7715e597 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:00.820740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:00.833327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6660b8-0ffe-4e03-8bb3-803e7715e597 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:00.838473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-eb6660b8-0ffe-4e03-8bb3-803e7715e597 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:00.838473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:00.838473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:00.838473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:00.838473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:00.838473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:00.838473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:00.838473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:00.838473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:00.838473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:00.838473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:00.838473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:00.838473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:00.838473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:00.838473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:00.838473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:00.839393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:00.839393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:00.839393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:00.839393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:00.839393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:00.839393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:00.839393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:00.839393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:00.839393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:00.839393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:00.839393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:00.839393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:00.839393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:00.839393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:00.839393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:00.839393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:00.840251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:00.840251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:00.840251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:00.840251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:00.840251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:00.840251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:00.840251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:00.840251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:00.840251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:00.840251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:00.840251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:00.840251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:00.840251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 126/252] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:00.842815 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6660b8-0ffe-4e03-8bb3-803e7715e597 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:00.923108 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18519c83-3844-40aa-be40-55ba2879e98a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:00.926686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:00.969447 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c963fec-9558-464a-b30f-57baebf6ff00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:00.970430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c963fec-9558-464a-b30f-57baebf6ff00 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:00.970941 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c963fec-9558-464a-b30f-57baebf6ff00 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:00.971414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c963fec-9558-464a-b30f-57baebf6ff00 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:00.982324 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c963fec-9558-464a-b30f-57baebf6ff00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:00.988003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-4c963fec-9558-464a-b30f-57baebf6ff00 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:08:00.988003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:00.988003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:00.988003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:00.988003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:00.988003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:00.988003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:00.988003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:00.988003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:00.988003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:00.988003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:00.988003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:00.988003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:00.988003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:00.988003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:00.988003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:00.988931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:00.988931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:00.988931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:00.988931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:00.988931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:00.988931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:00.988931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:00.988931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:00.988931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:00.988931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:00.988931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:00.988931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:00.988931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:00.988931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:00.988931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:00.988931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:00.989739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:00.989739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:00.989739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:00.989739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:00.989739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:00.989739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:00.989739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:00.989739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:00.989739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:00.989739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:00.989739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:08:00.989739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:00.993335 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 127/253] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:08:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:00.994910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c963fec-9558-464a-b30f-57baebf6ff00 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:00.997313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6660b8-0ffe-4e03-8bb3-803e7715e597 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:01.040432 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c9277-7a1c-40a3-af98-3a44c8859de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:01.041517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c9277-7a1c-40a3-af98-3a44c8859de5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:01.042029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c9277-7a1c-40a3-af98-3a44c8859de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:01.042503 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c9277-7a1c-40a3-af98-3a44c8859de5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:01.043428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c9277-7a1c-40a3-af98-3a44c8859de5 None admin] RBAC: Adding query filter params (role.id=061fff977e1f4d13a7b5e8e70edc3e92, scope.project.id=d844147ccee64e23b49af3bf86ac4c6f, user.id=ba2fc87b5e1a4d76976a702bdb6397e2) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:01.043980 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c9277-7a1c-40a3-af98-3a44c8859de5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:01.046249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c9277-7a1c-40a3-af98-3a44c8859de5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:01.054446 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 127/254] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:08:00 2022] GET /identity/v3/role_assignments?user.id=ba2fc87b5e1a4d76976a702bdb6397e2&scope.project.id=d844147ccee64e23b49af3bf86ac4c6f&role.id=061fff977e1f4d13a7b5e8e70edc3e92 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:01.055604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c9277-7a1c-40a3-af98-3a44c8859de5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:01.129727 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:01.153560 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c963fec-9558-464a-b30f-57baebf6ff00 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:01.183740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c9277-7a1c-40a3-af98-3a44c8859de5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:01.224740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424d6b7-aa4d-406f-aa6d-38397afd8253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:01.225780 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424d6b7-aa4d-406f-aa6d-38397afd8253 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:01.226286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424d6b7-aa4d-406f-aa6d-38397afd8253 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:01.226838 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424d6b7-aa4d-406f-aa6d-38397afd8253 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:01.239651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68daced-6136-4fc8-9b68-a5b131fe3831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:01.239651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68daced-6136-4fc8-9b68-a5b131fe3831 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:01.239651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68daced-6136-4fc8-9b68-a5b131fe3831 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:01.240006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68daced-6136-4fc8-9b68-a5b131fe3831 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:01.241625 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68daced-6136-4fc8-9b68-a5b131fe3831 None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:01.241625 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68daced-6136-4fc8-9b68-a5b131fe3831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:01.242156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68daced-6136-4fc8-9b68-a5b131fe3831 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:01.247682 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424d6b7-aa4d-406f-aa6d-38397afd8253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:01.257063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-a424d6b7-aa4d-406f-aa6d-38397afd8253 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:08:01.257063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:01.257063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:01.257063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:01.257063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:01.257063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:01.257063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:01.257063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:01.257063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:01.257063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:01.257063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:01.257063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:01.257063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:01.257063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:01.257063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:01.257063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:01.258045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:01.258045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:01.258045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:01.258045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:01.258045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:01.258045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:01.258045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:01.258045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:01.258045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:01.258045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:01.258045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:01.258045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:01.258045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:01.258045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:01.258045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:01.258045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:01.258898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:01.258898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:01.258898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:01.258898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:01.258898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:01.258898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:01.258898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:01.258898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:01.258898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:01.258898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:01.258898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:08:01.258898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:01.262118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 128/255] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:08:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:01.263231 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424d6b7-aa4d-406f-aa6d-38397afd8253 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:01.269523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 128/256] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:08:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:01.270394 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68daced-6136-4fc8-9b68-a5b131fe3831 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:01.321032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:01.383652 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424d6b7-aa4d-406f-aa6d-38397afd8253 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:01.402467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68daced-6136-4fc8-9b68-a5b131fe3831 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:01.403815 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:01.425761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34e12d-53fc-46a7-b3a6-bfdd8e918f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:01.426546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e12d-53fc-46a7-b3a6-bfdd8e918f1b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:01.426930 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e12d-53fc-46a7-b3a6-bfdd8e918f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:01.427306 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e12d-53fc-46a7-b3a6-bfdd8e918f1b None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:01.428149 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34e12d-53fc-46a7-b3a6-bfdd8e918f1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:01.428597 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34e12d-53fc-46a7-b3a6-bfdd8e918f1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:01.430094 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34e12d-53fc-46a7-b3a6-bfdd8e918f1b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:01.439994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 129/257] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:01.441324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34e12d-53fc-46a7-b3a6-bfdd8e918f1b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:01.496250 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edff7db-06f0-479f-876a-a9a5d843dfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:01.496885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edff7db-06f0-479f-876a-a9a5d843dfef None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:01.497167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edff7db-06f0-479f-876a-a9a5d843dfef None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:01.497437 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edff7db-06f0-479f-876a-a9a5d843dfef None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:01.498113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edff7db-06f0-479f-876a-a9a5d843dfef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:01.498417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edff7db-06f0-479f-876a-a9a5d843dfef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:01.499827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edff7db-06f0-479f-876a-a9a5d843dfef None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:01.533775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 129/258] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:01.534650 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edff7db-06f0-479f-876a-a9a5d843dfef None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:01.550353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34e12d-53fc-46a7-b3a6-bfdd8e918f1b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:01.588801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a555a-e2ac-40e2-be20-49f8c84dbcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:01.589591 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a555a-e2ac-40e2-be20-49f8c84dbcd7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:01.589980 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a555a-e2ac-40e2-be20-49f8c84dbcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:01.590384 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a555a-e2ac-40e2-be20-49f8c84dbcd7 None admin] PATH_INFO: `/v3/users/f46600603b394108b26ea29d87152d22` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:01.591713 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a555a-e2ac-40e2-be20-49f8c84dbcd7 None admin] RBAC: Authorizing `identity:get_user(user_id=f46600603b394108b26ea29d87152d22)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:01.593495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a555a-e2ac-40e2-be20-49f8c84dbcd7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:01.595704 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 130/259] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:08:01 2022] GET /identity/v3/users/f46600603b394108b26ea29d87152d22 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:01.596626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a555a-e2ac-40e2-be20-49f8c84dbcd7 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:01.671968 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:01.699014 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:01.706526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a555a-e2ac-40e2-be20-49f8c84dbcd7 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:01.740865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edff7db-06f0-479f-876a-a9a5d843dfef None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:01.747487 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9129a9f-9b04-4d0d-8951-8e1d42ca9f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:01.748268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9129a9f-9b04-4d0d-8951-8e1d42ca9f84 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:01.749492 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9129a9f-9b04-4d0d-8951-8e1d42ca9f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:01.749896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9129a9f-9b04-4d0d-8951-8e1d42ca9f84 None admin] PATH_INFO: `/v3/users/45a6bfb5faab4ef8aa91d24075adb297` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:01.751158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9129a9f-9b04-4d0d-8951-8e1d42ca9f84 None admin] RBAC: Authorizing `identity:get_user(user_id=45a6bfb5faab4ef8aa91d24075adb297)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:01.752780 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9129a9f-9b04-4d0d-8951-8e1d42ca9f84 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:01.754655 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 131/260] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:08:01 2022] GET /identity/v3/users/45a6bfb5faab4ef8aa91d24075adb297 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:01.755367 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9129a9f-9b04-4d0d-8951-8e1d42ca9f84 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:01.840210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a4e33-5784-4366-8a37-183846ca98e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:01.840864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a4e33-5784-4366-8a37-183846ca98e1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:01.841151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a4e33-5784-4366-8a37-183846ca98e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:01.841423 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a4e33-5784-4366-8a37-183846ca98e1 None admin] PATH_INFO: `/v3/users/45a6bfb5faab4ef8aa91d24075adb297` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:01.848189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a4e33-5784-4366-8a37-183846ca98e1 None admin] RBAC: Authorizing `identity:get_user(user_id=45a6bfb5faab4ef8aa91d24075adb297)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:01.849654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a4e33-5784-4366-8a37-183846ca98e1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:01.851252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 130/261] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:08:01 2022] GET /identity/v3/users/45a6bfb5faab4ef8aa91d24075adb297 => generated 283 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:01.852041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a4e33-5784-4366-8a37-183846ca98e1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:01.864288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9129a9f-9b04-4d0d-8951-8e1d42ca9f84 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:01.892074 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:01.903903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab135c9-b2a2-4ce0-aa6c-bbc319778a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:01.904675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab135c9-b2a2-4ce0-aa6c-bbc319778a2f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:01.905059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab135c9-b2a2-4ce0-aa6c-bbc319778a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:01.905456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab135c9-b2a2-4ce0-aa6c-bbc319778a2f None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:01.906956 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab135c9-b2a2-4ce0-aa6c-bbc319778a2f None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:01.908668 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab135c9-b2a2-4ce0-aa6c-bbc319778a2f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:01.910619 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 132/262] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:01 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:01.911342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab135c9-b2a2-4ce0-aa6c-bbc319778a2f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:02.022698 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab135c9-b2a2-4ce0-aa6c-bbc319778a2f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:02.061011 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b988aa-8180-4363-95a3-c6a3c35c8c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:02.061791 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b988aa-8180-4363-95a3-c6a3c35c8c32 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:02.062190 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b988aa-8180-4363-95a3-c6a3c35c8c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:02.062566 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b988aa-8180-4363-95a3-c6a3c35c8c32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:02.118200 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a4e33-5784-4366-8a37-183846ca98e1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:02.127934 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:02.140916 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b988aa-8180-4363-95a3-c6a3c35c8c32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:02.142908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b988aa-8180-4363-95a3-c6a3c35c8c32 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:02.188414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b57e4-8a41-49fe-83d7-6c04f3a5e581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:02.189198 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b57e4-8a41-49fe-83d7-6c04f3a5e581 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:02.189576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b57e4-8a41-49fe-83d7-6c04f3a5e581 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:02.189966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b57e4-8a41-49fe-83d7-6c04f3a5e581 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:02.219392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:02.280822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b57e4-8a41-49fe-83d7-6c04f3a5e581 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:02.282214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b57e4-8a41-49fe-83d7-6c04f3a5e581 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:02.354528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 133/263] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:02.355853 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b988aa-8180-4363-95a3-c6a3c35c8c32 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:02.395235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 131/264] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:02.396249 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b57e4-8a41-49fe-83d7-6c04f3a5e581 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:02.459721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:02.530338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b57e4-8a41-49fe-83d7-6c04f3a5e581 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:02.548842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:02.583423 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b988aa-8180-4363-95a3-c6a3c35c8c32 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:02.584648 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad5e19-7ec9-494a-8e1e-4ad909b12514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:02.585679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad5e19-7ec9-494a-8e1e-4ad909b12514 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:02.586157 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad5e19-7ec9-494a-8e1e-4ad909b12514 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:02.586626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad5e19-7ec9-494a-8e1e-4ad909b12514 None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:02.588555 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad5e19-7ec9-494a-8e1e-4ad909b12514 None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:02.590262 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad5e19-7ec9-494a-8e1e-4ad909b12514 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:02.592353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 132/265] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:02 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:02.593391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad5e19-7ec9-494a-8e1e-4ad909b12514 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:02.641542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63018db7-6a48-46ea-9351-cca67f6c6a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:02.642209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63018db7-6a48-46ea-9351-cca67f6c6a26 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:02.642482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63018db7-6a48-46ea-9351-cca67f6c6a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:02.642753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63018db7-6a48-46ea-9351-cca67f6c6a26 None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f/users/f46600603b394108b26ea29d87152d22/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:02.645567 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63018db7-6a48-46ea-9351-cca67f6c6a26 None admin] RBAC: Authorizing `identity:create_grant(project_id=d844147ccee64e23b49af3bf86ac4c6f, user_id=f46600603b394108b26ea29d87152d22, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:02.646940 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63018db7-6a48-46ea-9351-cca67f6c6a26 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:02.669892 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 134/266] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:08:02 2022] PUT /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f/users/f46600603b394108b26ea29d87152d22/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:08:02.673363 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63018db7-6a48-46ea-9351-cca67f6c6a26 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:02.766745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad5e19-7ec9-494a-8e1e-4ad909b12514 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:02.813513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:02.833469 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c4e37-e623-4209-bb63-c183d4cd890a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:02.834113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c4e37-e623-4209-bb63-c183d4cd890a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:02.834389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c4e37-e623-4209-bb63-c183d4cd890a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:02.834659 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c4e37-e623-4209-bb63-c183d4cd890a None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:02.835995 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c4e37-e623-4209-bb63-c183d4cd890a None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:02.837443 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c4e37-e623-4209-bb63-c183d4cd890a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:02.839291 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 133/267] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:02 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:02.845446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c4e37-e623-4209-bb63-c183d4cd890a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:02.860196 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63018db7-6a48-46ea-9351-cca67f6c6a26 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:02.923200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a7555-da20-4e31-a079-2a5de4b9c505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:02.923891 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a7555-da20-4e31-a079-2a5de4b9c505 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:02.924166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a7555-da20-4e31-a079-2a5de4b9c505 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:02.924433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a7555-da20-4e31-a079-2a5de4b9c505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:02.925158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a7555-da20-4e31-a079-2a5de4b9c505 None admin] RBAC: Adding query filter params (role.id=2c11a95c1ecf4988a95aad8d96ea56e9, scope.project.id=d844147ccee64e23b49af3bf86ac4c6f, user.id=45a6bfb5faab4ef8aa91d24075adb297) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:02.925466 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a7555-da20-4e31-a079-2a5de4b9c505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:02.926927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a7555-da20-4e31-a079-2a5de4b9c505 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:02.938910 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 135/268] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:08:02 2022] GET /identity/v3/role_assignments?user.id=45a6bfb5faab4ef8aa91d24075adb297&scope.project.id=d844147ccee64e23b49af3bf86ac4c6f&role.id=2c11a95c1ecf4988a95aad8d96ea56e9 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:03.020707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c4e37-e623-4209-bb63-c183d4cd890a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:03.071000 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec94c15-c4ca-4157-9d20-71234a5cf85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:03.071597 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec94c15-c4ca-4157-9d20-71234a5cf85f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:03.071853 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec94c15-c4ca-4157-9d20-71234a5cf85f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:03.072128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec94c15-c4ca-4157-9d20-71234a5cf85f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:03.073429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec94c15-c4ca-4157-9d20-71234a5cf85f None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=d844147ccee64e23b49af3bf86ac4c6f, user.id=45a6bfb5faab4ef8aa91d24075adb297) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:03.073744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec94c15-c4ca-4157-9d20-71234a5cf85f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:03.075252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec94c15-c4ca-4157-9d20-71234a5cf85f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:03.082242 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 134/269] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:08:02 2022] GET /identity/v3/role_assignments?user.id=45a6bfb5faab4ef8aa91d24075adb297&scope.project.id=d844147ccee64e23b49af3bf86ac4c6f&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:03.253653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee952e0-176e-4f9e-9e27-e4ddeb134330 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:03.254154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee952e0-176e-4f9e-9e27-e4ddeb134330 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:03.254228 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee952e0-176e-4f9e-9e27-e4ddeb134330 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:03.255552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 136/270] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:03.261471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1487f-e714-46c8-8d01-c73313e75699 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:03.261975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1487f-e714-46c8-8d01-c73313e75699 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:03.262051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1487f-e714-46c8-8d01-c73313e75699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:03.282381 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e6c1487f-e714-46c8-8d01-c73313e75699 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:03.283633 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e6c1487f-e714-46c8-8d01-c73313e75699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:03.317527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496a856-d343-4081-b079-f5c7924f562d None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:03.317838 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496a856-d343-4081-b079-f5c7924f562d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:03.318087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496a856-d343-4081-b079-f5c7924f562d None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:03.319793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 137/271] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:03.325832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc4ae8-2b25-4280-acd3-c49c8908d182 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:03.326108 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc4ae8-2b25-4280-acd3-c49c8908d182 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:03.326310 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc4ae8-2b25-4280-acd3-c49c8908d182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:03.346303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:03.361099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-5cdc4ae8-2b25-4280-acd3-c49c8908d182 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:03.362544 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5cdc4ae8-2b25-4280-acd3-c49c8908d182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:03.367164 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:03.367300 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 135/272] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:03.462364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 138/273] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:03.675928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1487f-e714-46c8-8d01-c73313e75699 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:03.772200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc4ae8-2b25-4280-acd3-c49c8908d182 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:03.779279 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1487f-e714-46c8-8d01-c73313e75699 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:03.817716 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf9dec-0bc8-41ee-9ac9-34c448fc471f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:03.818364 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf9dec-0bc8-41ee-9ac9-34c448fc471f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:03.818651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf9dec-0bc8-41ee-9ac9-34c448fc471f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:03.818926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf9dec-0bc8-41ee-9ac9-34c448fc471f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:03.820097 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf9dec-0bc8-41ee-9ac9-34c448fc471f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:03.821601 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf9dec-0bc8-41ee-9ac9-34c448fc471f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:03.824067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 136/274] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:03.830700 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf9dec-0bc8-41ee-9ac9-34c448fc471f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:03.895825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc4ae8-2b25-4280-acd3-c49c8908d182 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:03.908214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:03.940654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039fc08-ea37-4c6a-8f56-18e00b49dbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:03.941362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039fc08-ea37-4c6a-8f56-18e00b49dbae None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:03.941643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039fc08-ea37-4c6a-8f56-18e00b49dbae None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:03.941916 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039fc08-ea37-4c6a-8f56-18e00b49dbae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:03.949591 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:03.952370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039fc08-ea37-4c6a-8f56-18e00b49dbae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:03.957628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-7039fc08-ea37-4c6a-8f56-18e00b49dbae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:03.957628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:03.957628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:03.957628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:03.957628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:03.957628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:03.957628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:03.957628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:03.957628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:03.957628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:03.957628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:03.957628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:03.957628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:03.957628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:03.957628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:03.957628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:03.958746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:03.958746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:03.958746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:03.958746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:03.958746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:03.958746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:03.958746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:03.958746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:03.958746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:03.958746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:03.958746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:03.958746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:03.958746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:03.958746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:03.958746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:03.958746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:03.968226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:03.968226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:03.968226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:03.968226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:03.968226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:03.968226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:03.968226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:03.968226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:03.968226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:03.968226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:03.968226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:03.968226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:03.968226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 139/275] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:08:03.968226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173066c-acaf-482e-b2d6-5cbf7fbaac4f None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:03.969034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173066c-acaf-482e-b2d6-5cbf7fbaac4f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:03.969111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf9dec-0bc8-41ee-9ac9-34c448fc471f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:03.969184 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173066c-acaf-482e-b2d6-5cbf7fbaac4f None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:03.969184 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 140/276] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:03.969184 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173066c-acaf-482e-b2d6-5cbf7fbaac4f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:03.996940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc09ddd-2e56-4698-bd54-eb0c7e40ef06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:03.997662 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc09ddd-2e56-4698-bd54-eb0c7e40ef06 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:03.997854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc09ddd-2e56-4698-bd54-eb0c7e40ef06 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:03.998126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc09ddd-2e56-4698-bd54-eb0c7e40ef06 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:03.998777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc09ddd-2e56-4698-bd54-eb0c7e40ef06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:04.000258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc09ddd-2e56-4698-bd54-eb0c7e40ef06 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:04.022677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 137/277] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:08:03 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:08:04.024977 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e9db8-2c65-4e4c-8c66-54e8c2ad24c8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:04.025346 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e9db8-2c65-4e4c-8c66-54e8c2ad24c8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:04.026263 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e9db8-2c65-4e4c-8c66-54e8c2ad24c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:04.055549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-af5e9db8-2c65-4e4c-8c66-54e8c2ad24c8 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:04.057694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-af5e9db8-2c65-4e4c-8c66-54e8c2ad24c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:04.089227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173066c-acaf-482e-b2d6-5cbf7fbaac4f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:04.131561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ce1f8-4ab9-4abb-bed9-c23ad9801c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:04.132483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ce1f8-4ab9-4abb-bed9-c23ad9801c78 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:04.133131 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ce1f8-4ab9-4abb-bed9-c23ad9801c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:04.133714 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ce1f8-4ab9-4abb-bed9-c23ad9801c78 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:04.134635 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ce1f8-4ab9-4abb-bed9-c23ad9801c78 None admin] RBAC: Adding query filter params (name=member) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:04.135247 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ce1f8-4ab9-4abb-bed9-c23ad9801c78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:04.137074 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ce1f8-4ab9-4abb-bed9-c23ad9801c78 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:04.148825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 141/278] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:08:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:04.154895 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ce1f8-4ab9-4abb-bed9-c23ad9801c78 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:04.161684 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 138/279] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:04.223024 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:04.277573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ce1f8-4ab9-4abb-bed9-c23ad9801c78 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:04.314542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72434547-9a19-4aef-88cb-d275c893b9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:04.315180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72434547-9a19-4aef-88cb-d275c893b9eb None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:04.315598 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72434547-9a19-4aef-88cb-d275c893b9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:04.315936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72434547-9a19-4aef-88cb-d275c893b9eb None admin] PATH_INFO: `/v3/users/576770327b8841eda5056fbe7e49579a` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:04.331412 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72434547-9a19-4aef-88cb-d275c893b9eb None admin] RBAC: Authorizing `identity:get_user(user_id=576770327b8841eda5056fbe7e49579a)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:04.332928 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72434547-9a19-4aef-88cb-d275c893b9eb None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:04.334765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 142/280] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:08:04 2022] GET /identity/v3/users/576770327b8841eda5056fbe7e49579a => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:04.340913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e9db8-2c65-4e4c-8c66-54e8c2ad24c8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:04.385707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:04.435680 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72434547-9a19-4aef-88cb-d275c893b9eb None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:04.464333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e9db8-2c65-4e4c-8c66-54e8c2ad24c8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:04.506242 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b8aa8-972d-4ada-8f43-abd5960ba7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:04.506781 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b8aa8-972d-4ada-8f43-abd5960ba7db None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:04.507061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b8aa8-972d-4ada-8f43-abd5960ba7db None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:04.507331 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b8aa8-972d-4ada-8f43-abd5960ba7db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:04.508168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b8aa8-972d-4ada-8f43-abd5960ba7db None admin] RBAC: Adding query filter params (role.id=bbea734b8a704d7ca8ec61bd45536cbe, user.id=576770327b8841eda5056fbe7e49579a, scope.system=all) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:04.508603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b8aa8-972d-4ada-8f43-abd5960ba7db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:04.510850 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b8aa8-972d-4ada-8f43-abd5960ba7db None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:04.516432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 139/281] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Oct 6 12:08:04 2022] GET /identity/v3/role_assignments?user.id=576770327b8841eda5056fbe7e49579a&scope.system=all&role.id=bbea734b8a704d7ca8ec61bd45536cbe => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:04.587955 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72434547-9a19-4aef-88cb-d275c893b9eb None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:04.617232 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:04.637893 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4d56a-ed5d-48c3-8679-bad8d4730e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:04.638547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4d56a-ed5d-48c3-8679-bad8d4730e6e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:04.638825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4d56a-ed5d-48c3-8679-bad8d4730e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:04.639095 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4d56a-ed5d-48c3-8679-bad8d4730e6e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:04.651391 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4d56a-ed5d-48c3-8679-bad8d4730e6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:04.656836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-3ab4d56a-ed5d-48c3-8679-bad8d4730e6e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:08:04.656836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:04.656836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:04.656836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:04.656836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:04.656836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:04.656836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:04.656836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:04.656836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:04.656836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:04.656836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:04.656836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:04.656836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:04.656836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:04.656836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:04.656836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:04.657807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:04.657807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:04.657807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:04.657807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:04.657807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:04.657807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:04.657807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:04.657807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:04.657807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:04.657807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:04.657807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:04.657807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:04.657807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:04.657807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:04.657807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:04.657807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:04.659039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:04.659039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:04.659039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:04.659039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:04.659039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:04.659039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:04.659039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:04.659039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:04.659039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:04.659039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:04.659039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:08:04.659039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:04.659039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 143/282] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:08:04.664005 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b8aa8-972d-4ada-8f43-abd5960ba7db None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:04.820294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b8aa8-972d-4ada-8f43-abd5960ba7db None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:04.822677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:04.855948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0f34a-1a14-418a-82d8-1532c11d8d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:04.856562 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0f34a-1a14-418a-82d8-1532c11d8d27 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:04.856836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0f34a-1a14-418a-82d8-1532c11d8d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:04.857102 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0f34a-1a14-418a-82d8-1532c11d8d27 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:04.857762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0f34a-1a14-418a-82d8-1532c11d8d27 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:04.858058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0f34a-1a14-418a-82d8-1532c11d8d27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:04.859424 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0f34a-1a14-418a-82d8-1532c11d8d27 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:04.868303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 140/283] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:08:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:04.874286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4d56a-ed5d-48c3-8679-bad8d4730e6e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:05.024028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4d56a-ed5d-48c3-8679-bad8d4730e6e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:05.081234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794634b-6a6f-461d-bada-6b0522ca1673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:05.081800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794634b-6a6f-461d-bada-6b0522ca1673 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:05.082085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794634b-6a6f-461d-bada-6b0522ca1673 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:05.082363 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794634b-6a6f-461d-bada-6b0522ca1673 None admin] PATH_INFO: `/v3/users/ba2fc87b5e1a4d76976a702bdb6397e2` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:05.083533 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794634b-6a6f-461d-bada-6b0522ca1673 None admin] RBAC: Authorizing `identity:get_user(user_id=ba2fc87b5e1a4d76976a702bdb6397e2)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:05.085006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794634b-6a6f-461d-bada-6b0522ca1673 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:05.086875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 144/284] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:08:04 2022] GET /identity/v3/users/ba2fc87b5e1a4d76976a702bdb6397e2 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:05.092620 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0f34a-1a14-418a-82d8-1532c11d8d27 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:05.220213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0f34a-1a14-418a-82d8-1532c11d8d27 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:05.257323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6117e-2ce3-4806-8bc7-eb82d6be672a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:05.258050 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6117e-2ce3-4806-8bc7-eb82d6be672a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:05.258328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6117e-2ce3-4806-8bc7-eb82d6be672a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:05.258598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6117e-2ce3-4806-8bc7-eb82d6be672a None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:05.260431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6117e-2ce3-4806-8bc7-eb82d6be672a None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:05.261932 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6117e-2ce3-4806-8bc7-eb82d6be672a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:05.263946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 141/285] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:05 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:05.269442 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794634b-6a6f-461d-bada-6b0522ca1673 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:05.318217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:05.391707 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794634b-6a6f-461d-bada-6b0522ca1673 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:05.439187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3c7bb-93d8-469c-8a69-8a664c568cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:05.439887 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3c7bb-93d8-469c-8a69-8a664c568cac None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:05.440165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3c7bb-93d8-469c-8a69-8a664c568cac None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:05.440510 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3c7bb-93d8-469c-8a69-8a664c568cac None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f/users/ba2fc87b5e1a4d76976a702bdb6397e2/roles/061fff977e1f4d13a7b5e8e70edc3e92` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:05.444849 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3c7bb-93d8-469c-8a69-8a664c568cac None admin] RBAC: Authorizing `identity:create_grant(project_id=d844147ccee64e23b49af3bf86ac4c6f, user_id=ba2fc87b5e1a4d76976a702bdb6397e2, role_id=061fff977e1f4d13a7b5e8e70edc3e92)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:05.446357 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3c7bb-93d8-469c-8a69-8a664c568cac None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:05.470351 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 145/286] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:08:05 2022] PUT /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f/users/ba2fc87b5e1a4d76976a702bdb6397e2/roles/061fff977e1f4d13a7b5e8e70edc3e92 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:08:05.610931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57c2ba-4cc8-422a-9a4d-cb609d12e1ea None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:05.611446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57c2ba-4cc8-422a-9a4d-cb609d12e1ea None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:05.611446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57c2ba-4cc8-422a-9a4d-cb609d12e1ea None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:05.612945 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 142/287] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:05.618519 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ed0a4-213a-4733-b3d7-019ded162606 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:05.618797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ed0a4-213a-4733-b3d7-019ded162606 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:05.619600 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ed0a4-213a-4733-b3d7-019ded162606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:05.644696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-008ed0a4-213a-4733-b3d7-019ded162606 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:05.645816 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-008ed0a4-213a-4733-b3d7-019ded162606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:05.736247 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 146/288] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:06.099667 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57c2ba-4cc8-422a-9a4d-cb609d12e1ea None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:06.141831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:06.229191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57c2ba-4cc8-422a-9a4d-cb609d12e1ea None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:06.263353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62c774-07d5-410b-9e46-e0635a5c493f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:06.264012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62c774-07d5-410b-9e46-e0635a5c493f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:06.264287 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62c774-07d5-410b-9e46-e0635a5c493f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:06.264554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62c774-07d5-410b-9e46-e0635a5c493f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:06.274844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62c774-07d5-410b-9e46-e0635a5c493f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:06.280125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-be62c774-07d5-410b-9e46-e0635a5c493f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:06.280125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:06.280125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:06.280125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:06.280125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:06.280125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:06.280125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:06.280125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:06.280125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:06.280125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:06.280125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:06.280125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:06.280125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:06.280125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:06.280125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:06.280125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:06.281264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:06.281264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:06.281264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:06.281264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:06.281264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:06.281264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:06.281264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:06.281264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:06.281264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:06.281264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:06.281264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:06.281264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:06.281264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:06.281264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:06.281264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:06.281264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:06.282079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:06.282079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:06.282079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:06.282079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:06.282079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:06.282079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:06.282079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:06.282079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:06.282079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:06.282079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:06.282079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:06.282079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:06.282079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 143/289] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:06.287789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ed0a4-213a-4733-b3d7-019ded162606 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:06.353869 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:06.454226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ed0a4-213a-4733-b3d7-019ded162606 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:06.497765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c46e21-a7fb-4b1a-9002-86974603f530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:06.498665 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c46e21-a7fb-4b1a-9002-86974603f530 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:06.499121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c46e21-a7fb-4b1a-9002-86974603f530 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:06.499589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c46e21-a7fb-4b1a-9002-86974603f530 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:06.500450 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c46e21-a7fb-4b1a-9002-86974603f530 None admin] RBAC: Adding query filter params (name=member) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:06.500925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c46e21-a7fb-4b1a-9002-86974603f530 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:06.502460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c46e21-a7fb-4b1a-9002-86974603f530 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:06.511210 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 147/290] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:08:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:06.516571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62c774-07d5-410b-9e46-e0635a5c493f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:06.647173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:06.653919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62c774-07d5-410b-9e46-e0635a5c493f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:06.690146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc14da-f188-4130-bc24-bc6ea670891a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:06.690808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc14da-f188-4130-bc24-bc6ea670891a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:06.691091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc14da-f188-4130-bc24-bc6ea670891a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:06.691359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc14da-f188-4130-bc24-bc6ea670891a None admin] PATH_INFO: `/v3/users/f46600603b394108b26ea29d87152d22` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:06.692679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc14da-f188-4130-bc24-bc6ea670891a None admin] RBAC: Authorizing `identity:get_user(user_id=f46600603b394108b26ea29d87152d22)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:06.694304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc14da-f188-4130-bc24-bc6ea670891a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:06.696389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 144/291] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:08:06 2022] GET /identity/v3/users/f46600603b394108b26ea29d87152d22 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:06.702773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c46e21-a7fb-4b1a-9002-86974603f530 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:06.824787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:06.829193 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c46e21-a7fb-4b1a-9002-86974603f530 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:06.878738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fecae8-2052-45db-b589-7430f192cdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:06.879376 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fecae8-2052-45db-b589-7430f192cdd9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:06.879759 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fecae8-2052-45db-b589-7430f192cdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:06.880034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fecae8-2052-45db-b589-7430f192cdd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:06.889429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bc3ae-2992-4cf9-8276-639cf829dc35 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:06.889753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bc3ae-2992-4cf9-8276-639cf829dc35 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:06.889988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bc3ae-2992-4cf9-8276-639cf829dc35 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:06.891629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 145/292] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:06.897247 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73820021-b33b-4c3d-b4e3-a1aad070b8c2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:06.897505 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73820021-b33b-4c3d-b4e3-a1aad070b8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:06.897724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73820021-b33b-4c3d-b4e3-a1aad070b8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:06.917044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-73820021-b33b-4c3d-b4e3-a1aad070b8c2 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:06.918234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-73820021-b33b-4c3d-b4e3-a1aad070b8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:06.991402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 146/293] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:08:07.006547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fecae8-2052-45db-b589-7430f192cdd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:07.008052 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fecae8-2052-45db-b589-7430f192cdd9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:07.113248 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c34a39-3a2e-452d-9e00-b8139da2197e None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:07.113526 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c34a39-3a2e-452d-9e00-b8139da2197e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:07.114029 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c34a39-3a2e-452d-9e00-b8139da2197e None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:07.117126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 147/294] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:08:07.121577 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2ed07-94ad-4ebd-95fb-72453f97d345 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:07.122127 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2ed07-94ad-4ebd-95fb-72453f97d345 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:07.122604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2ed07-94ad-4ebd-95fb-72453f97d345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:07.146535 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:07.149724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9fa2ed07-94ad-4ebd-95fb-72453f97d345 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:07.152668 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9fa2ed07-94ad-4ebd-95fb-72453f97d345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:07.159532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 148/295] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:07.165036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fecae8-2052-45db-b589-7430f192cdd9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:07.291034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 148/296] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:07.312750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa2ed07-94ad-4ebd-95fb-72453f97d345 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:07.343866 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:07.350300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fecae8-2052-45db-b589-7430f192cdd9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:07.398252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600f0d2-fb5a-42ee-8d21-566ac30715c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:07.398902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600f0d2-fb5a-42ee-8d21-566ac30715c5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:07.399174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600f0d2-fb5a-42ee-8d21-566ac30715c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:07.399421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600f0d2-fb5a-42ee-8d21-566ac30715c5 None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:07.400902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600f0d2-fb5a-42ee-8d21-566ac30715c5 None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:07.402289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600f0d2-fb5a-42ee-8d21-566ac30715c5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:07.404298 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 149/297] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:07 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:07.405775 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03df53-f17a-40de-952d-50f0e496d429 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:07.406062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03df53-f17a-40de-952d-50f0e496d429 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:07.406334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03df53-f17a-40de-952d-50f0e496d429 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:07.411712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 150/298] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:07.412592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03df53-f17a-40de-952d-50f0e496d429 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:07.436173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa2ed07-94ad-4ebd-95fb-72453f97d345 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:07.470292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbbda6-1cac-4798-9f87-8ac946d60eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:07.470978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbbda6-1cac-4798-9f87-8ac946d60eee None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:07.471259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbbda6-1cac-4798-9f87-8ac946d60eee None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:07.471554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbbda6-1cac-4798-9f87-8ac946d60eee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:07.482271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbbda6-1cac-4798-9f87-8ac946d60eee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:07.484494 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:07.487137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-4bdbbda6-1cac-4798-9f87-8ac946d60eee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:08:07.487137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:07.487137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:07.487137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:07.487137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:07.487137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:07.487137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:07.487137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:07.487137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:07.487137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:07.487137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:07.487137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:07.487137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:07.487137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:07.487137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:07.487137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:07.488066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:07.488066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:07.488066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:07.488066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:07.488066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:07.488066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:07.488066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:07.488066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:07.488066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:07.488066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:07.488066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:07.488066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:07.488066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:07.488066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:07.488066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:07.488066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:07.488894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:07.488894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:07.488894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:07.488894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:07.488894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:07.488894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:07.488894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:07.488894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:07.488894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:07.488894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:07.488894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:08:07.488894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:07.493133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 149/299] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:08:07.494955 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967a7ac-07df-44b7-928d-74b7cc4b9cb6 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:07.495262 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967a7ac-07df-44b7-928d-74b7cc4b9cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:07.495530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967a7ac-07df-44b7-928d-74b7cc4b9cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:07.513335 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-4967a7ac-07df-44b7-928d-74b7cc4b9cb6 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:07.514501 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4967a7ac-07df-44b7-928d-74b7cc4b9cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:07.521099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03df53-f17a-40de-952d-50f0e496d429 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:07.557244 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3dce8-b1cc-4fb2-973f-8237df413322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:07.557944 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3dce8-b1cc-4fb2-973f-8237df413322 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:07.558228 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3dce8-b1cc-4fb2-973f-8237df413322 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:07.558501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3dce8-b1cc-4fb2-973f-8237df413322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:07.559332 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3dce8-b1cc-4fb2-973f-8237df413322 None admin] RBAC: Adding query filter params (role.id=bbea734b8a704d7ca8ec61bd45536cbe, scope.project.id=d844147ccee64e23b49af3bf86ac4c6f, user.id=f46600603b394108b26ea29d87152d22) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:07.559688 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3dce8-b1cc-4fb2-973f-8237df413322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:07.561113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3dce8-b1cc-4fb2-973f-8237df413322 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:07.567463 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 151/300] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:08:07 2022] GET /identity/v3/role_assignments?user.id=f46600603b394108b26ea29d87152d22&scope.project.id=d844147ccee64e23b49af3bf86ac4c6f&role.id=bbea734b8a704d7ca8ec61bd45536cbe => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:07.568212 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3dce8-b1cc-4fb2-973f-8237df413322 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:07.602313 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 150/301] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:07.620038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967a7ac-07df-44b7-928d-74b7cc4b9cb6 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:07.636654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:07.742867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967a7ac-07df-44b7-928d-74b7cc4b9cb6 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:07.748627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3dce8-b1cc-4fb2-973f-8237df413322 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:07.792239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:07.798840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e46989-666c-40fa-8de4-a574e940bbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:07.799945 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e46989-666c-40fa-8de4-a574e940bbe1 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:07.800567 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e46989-666c-40fa-8de4-a574e940bbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:07.801013 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e46989-666c-40fa-8de4-a574e940bbe1 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:07.802553 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e46989-666c-40fa-8de4-a574e940bbe1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:07.802679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e159f-d302-4ed7-835f-262ea3bbbdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:07.803058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e159f-d302-4ed7-835f-262ea3bbbdad None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:07.803370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e46989-666c-40fa-8de4-a574e940bbe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:07.803471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e159f-d302-4ed7-835f-262ea3bbbdad None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:07.803781 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e159f-d302-4ed7-835f-262ea3bbbdad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:07.804949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e159f-d302-4ed7-835f-262ea3bbbdad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:07.805572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e46989-666c-40fa-8de4-a574e940bbe1 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:07.806546 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e159f-d302-4ed7-835f-262ea3bbbdad None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:07.811551 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 151/302] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:07.815307 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e159f-d302-4ed7-835f-262ea3bbbdad None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:07.824279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 152/303] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:08:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:07.830962 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e46989-666c-40fa-8de4-a574e940bbe1 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:07.912906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:07.939413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e159f-d302-4ed7-835f-262ea3bbbdad None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:07.974360 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e46989-666c-40fa-8de4-a574e940bbe1 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:07.982161 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33a406-6e25-4767-8b87-0d2a479d6697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:07.983409 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33a406-6e25-4767-8b87-0d2a479d6697 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:07.983921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33a406-6e25-4767-8b87-0d2a479d6697 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:07.984389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33a406-6e25-4767-8b87-0d2a479d6697 None admin] PATH_INFO: `/v3/groups` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:07.985223 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33a406-6e25-4767-8b87-0d2a479d6697 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:07.986867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33a406-6e25-4767-8b87-0d2a479d6697 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:08.006110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 152/304] 10.222.0.33 () {60 vars in 1134 bytes} [Thu Oct 6 12:08:07 2022] POST /identity/v3/groups => generated 229 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:08:08.006869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33a406-6e25-4767-8b87-0d2a479d6697 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:08.019436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4cffc1-096c-49ac-9a28-6712230b498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:08.020121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cffc1-096c-49ac-9a28-6712230b498f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:08.020398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cffc1-096c-49ac-9a28-6712230b498f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:08.020667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cffc1-096c-49ac-9a28-6712230b498f None admin] PATH_INFO: `/v3/users/03acf4a0d6b2488783e6071282f93c67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:08.039874 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cffc1-096c-49ac-9a28-6712230b498f None admin] RBAC: Authorizing `identity:get_user(user_id=03acf4a0d6b2488783e6071282f93c67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:08.041620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cffc1-096c-49ac-9a28-6712230b498f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:08.049275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 153/305] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:08:07 2022] GET /identity/v3/users/03acf4a0d6b2488783e6071282f93c67 => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:08.050464 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4cffc1-096c-49ac-9a28-6712230b498f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:08.109517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:08.173348 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33a406-6e25-4767-8b87-0d2a479d6697 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:08.180971 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4cffc1-096c-49ac-9a28-6712230b498f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:08.214811 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebc265-0344-40f7-be0e-6d55917ecd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:08.215448 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebc265-0344-40f7-be0e-6d55917ecd34 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:08.215758 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebc265-0344-40f7-be0e-6d55917ecd34 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:08.216027 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebc265-0344-40f7-be0e-6d55917ecd34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:08.227265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:08.232353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ebc265-0344-40f7-be0e-6d55917ecd34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:08.234332 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e5bcc-13ec-4901-b000-74167cba4e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:08.234944 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e5bcc-13ec-4901-b000-74167cba4e09 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:08.235222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e5bcc-13ec-4901-b000-74167cba4e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:08.235491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e5bcc-13ec-4901-b000-74167cba4e09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:08.236238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e5bcc-13ec-4901-b000-74167cba4e09 None admin] RBAC: Adding query filter params (role.id=061fff977e1f4d13a7b5e8e70edc3e92, user.id=03acf4a0d6b2488783e6071282f93c67, scope.system=all) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:08.236544 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e5bcc-13ec-4901-b000-74167cba4e09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:08.237964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e5bcc-13ec-4901-b000-74167cba4e09 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:08.239870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-32ebc265-0344-40f7-be0e-6d55917ecd34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:08.239870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:08.239870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:08.239870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:08.239870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:08.239870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:08.239870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:08.239870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:08.239870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:08.239870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:08.239870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:08.239870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:08.239870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:08.239870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:08.239870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:08.239870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:08.241186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:08.241186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:08.241186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:08.241186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:08.241186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:08.241186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:08.241186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:08.241186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:08.241186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:08.241186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:08.241186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:08.241186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:08.241186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:08.241186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:08.241186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:08.241186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:08.242004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:08.242004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:08.242004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:08.242004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:08.242004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:08.242004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:08.242004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:08.242004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:08.242004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:08.242004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:08.242004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:08.242004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:08.244912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 154/306] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Oct 6 12:08:08 2022] GET /identity/v3/role_assignments?user.id=03acf4a0d6b2488783e6071282f93c67&scope.system=all&role.id=061fff977e1f4d13a7b5e8e70edc3e92 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:08.245548 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 153/307] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:08.252650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e5bcc-13ec-4901-b000-74167cba4e09 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:08.280468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0c974-ddee-49a3-ac46-fe19c3c37a3f None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:08.281250 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0c974-ddee-49a3-ac46-fe19c3c37a3f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:08.282193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0c974-ddee-49a3-ac46-fe19c3c37a3f None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:08.284200 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 154/308] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:08:08.290197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a52bd-0d85-47e4-bceb-e0f5d17e3f07 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:08.290794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a52bd-0d85-47e4-bceb-e0f5d17e3f07 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:08.291831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a52bd-0d85-47e4-bceb-e0f5d17e3f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:08.314468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-5a2a52bd-0d85-47e4-bceb-e0f5d17e3f07 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:08.316858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5a2a52bd-0d85-47e4-bceb-e0f5d17e3f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:08.390421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e5bcc-13ec-4901-b000-74167cba4e09 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:08.411773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:08.415300 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 155/309] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:08:08.442534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc43d0-b089-4cd4-b7f3-666e53facd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:08.443147 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc43d0-b089-4cd4-b7f3-666e53facd9d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:08.443426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc43d0-b089-4cd4-b7f3-666e53facd9d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:08.443801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc43d0-b089-4cd4-b7f3-666e53facd9d None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:08.444467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc43d0-b089-4cd4-b7f3-666e53facd9d None admin] RBAC: Adding query filter params (name=member) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:08.444803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc43d0-b089-4cd4-b7f3-666e53facd9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:08.446134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc43d0-b089-4cd4-b7f3-666e53facd9d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:08.455545 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 155/310] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:08:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:08.464680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a52bd-0d85-47e4-bceb-e0f5d17e3f07 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:08.569832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a52bd-0d85-47e4-bceb-e0f5d17e3f07 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:08.600542 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:08.607341 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a123c-4bcd-482a-bc94-25c6223b7bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:08.608005 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a123c-4bcd-482a-bc94-25c6223b7bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:08.608286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a123c-4bcd-482a-bc94-25c6223b7bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:08.608559 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a123c-4bcd-482a-bc94-25c6223b7bf7 None admin] PATH_INFO: `/v3/users/576770327b8841eda5056fbe7e49579a` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:08.609697 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a123c-4bcd-482a-bc94-25c6223b7bf7 None admin] RBAC: Authorizing `identity:get_user(user_id=576770327b8841eda5056fbe7e49579a)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:08.611146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a123c-4bcd-482a-bc94-25c6223b7bf7 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:08.617363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 156/311] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:08:08 2022] GET /identity/v3/users/576770327b8841eda5056fbe7e49579a => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:08.618692 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc43d0-b089-4cd4-b7f3-666e53facd9d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:08.688595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a123c-4bcd-482a-bc94-25c6223b7bf7 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:08.729802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc43d0-b089-4cd4-b7f3-666e53facd9d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:08.771693 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a48dbe-8e21-40e9-8c94-005efcacdada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:08.772354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a48dbe-8e21-40e9-8c94-005efcacdada None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:08.772637 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a48dbe-8e21-40e9-8c94-005efcacdada None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:08.772913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a48dbe-8e21-40e9-8c94-005efcacdada None admin] PATH_INFO: `/v3/system/users/576770327b8841eda5056fbe7e49579a/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:08.774359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a48dbe-8e21-40e9-8c94-005efcacdada None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=576770327b8841eda5056fbe7e49579a, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:08.776303 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a48dbe-8e21-40e9-8c94-005efcacdada None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:08.787541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 156/312] 10.222.0.33 () {58 vars in 1334 bytes} [Thu Oct 6 12:08:08 2022] PUT /identity/v3/system/users/576770327b8841eda5056fbe7e49579a/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:08:08.819488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a123c-4bcd-482a-bc94-25c6223b7bf7 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:08.860241 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c76833-f3e6-4541-ae0e-a4ed2b3dc756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:08.860900 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c76833-f3e6-4541-ae0e-a4ed2b3dc756 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:08.861185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c76833-f3e6-4541-ae0e-a4ed2b3dc756 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:08.861461 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c76833-f3e6-4541-ae0e-a4ed2b3dc756 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:08.873986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c76833-f3e6-4541-ae0e-a4ed2b3dc756 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:08.879994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-32c76833-f3e6-4541-ae0e-a4ed2b3dc756 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:08:08.879994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:08.879994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:08.879994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:08.879994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:08.879994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:08.879994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:08.879994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:08.879994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:08.879994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:08.879994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:08.879994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:08.879994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:08.879994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:08.879994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:08.879994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:08.880967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:08.880967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:08.880967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:08.880967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:08.880967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:08.880967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:08.880967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:08.880967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:08.880967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:08.880967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:08.880967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:08.880967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:08.880967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:08.880967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:08.880967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:08.880967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:08.881769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:08.881769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:08.881769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:08.881769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:08.881769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:08.881769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:08.881769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:08.881769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:08.881769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:08.881769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:08.881769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:08:08.881769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:08.881769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 157/313] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:08.889538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a48dbe-8e21-40e9-8c94-005efcacdada None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:08.949061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:09.017001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a48dbe-8e21-40e9-8c94-005efcacdada None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:09.053056 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93516fd1-5520-45aa-965e-651dbbdc9ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:09.053961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93516fd1-5520-45aa-965e-651dbbdc9ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:09.054449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93516fd1-5520-45aa-965e-651dbbdc9ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:09.054902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93516fd1-5520-45aa-965e-651dbbdc9ed2 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:09.055762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93516fd1-5520-45aa-965e-651dbbdc9ed2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:09.056244 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93516fd1-5520-45aa-965e-651dbbdc9ed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:09.058341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93516fd1-5520-45aa-965e-651dbbdc9ed2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:09.067144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 157/314] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:08:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:09.073150 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c76833-f3e6-4541-ae0e-a4ed2b3dc756 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:09.135305 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:09.179079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c76833-f3e6-4541-ae0e-a4ed2b3dc756 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:09.215191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45577fab-01df-4bb0-bb11-ed4efe80d354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:09.216201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45577fab-01df-4bb0-bb11-ed4efe80d354 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:09.216744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45577fab-01df-4bb0-bb11-ed4efe80d354 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:09.217195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45577fab-01df-4bb0-bb11-ed4efe80d354 None admin] PATH_INFO: `/v3/users/ba2fc87b5e1a4d76976a702bdb6397e2` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:09.218639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45577fab-01df-4bb0-bb11-ed4efe80d354 None admin] RBAC: Authorizing `identity:get_user(user_id=ba2fc87b5e1a4d76976a702bdb6397e2)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:09.220350 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45577fab-01df-4bb0-bb11-ed4efe80d354 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:09.222614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 158/315] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:08:09 2022] GET /identity/v3/users/ba2fc87b5e1a4d76976a702bdb6397e2 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:09.228552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93516fd1-5520-45aa-965e-651dbbdc9ed2 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:09.343686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93516fd1-5520-45aa-965e-651dbbdc9ed2 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:09.345211 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:09.379542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4461c-882d-4c01-8536-bd67aab4cd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:09.380165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4461c-882d-4c01-8536-bd67aab4cd1c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:09.380455 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4461c-882d-4c01-8536-bd67aab4cd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:09.380724 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4461c-882d-4c01-8536-bd67aab4cd1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:09.448790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4461c-882d-4c01-8536-bd67aab4cd1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:09.450252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4461c-882d-4c01-8536-bd67aab4cd1c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:09.552848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 158/316] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:09.559269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45577fab-01df-4bb0-bb11-ed4efe80d354 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:09.660411 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45577fab-01df-4bb0-bb11-ed4efe80d354 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:09.685672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:09.694466 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83211cbe-491e-4195-8816-1d688b424ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:09.695096 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83211cbe-491e-4195-8816-1d688b424ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:09.695379 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83211cbe-491e-4195-8816-1d688b424ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:09.695684 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83211cbe-491e-4195-8816-1d688b424ee0 None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:09.696804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83211cbe-491e-4195-8816-1d688b424ee0 None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:09.698764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83211cbe-491e-4195-8816-1d688b424ee0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:09.700628 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 159/317] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:09 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:09.706290 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4461c-882d-4c01-8536-bd67aab4cd1c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:09.708505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:09.815796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4461c-882d-4c01-8536-bd67aab4cd1c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:09.850326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44331d3-d50c-40f8-b50b-b2444dc546c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:09.850940 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44331d3-d50c-40f8-b50b-b2444dc546c6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:09.851223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44331d3-d50c-40f8-b50b-b2444dc546c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:09.851522 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44331d3-d50c-40f8-b50b-b2444dc546c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:09.852248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44331d3-d50c-40f8-b50b-b2444dc546c6 None admin] RBAC: Adding query filter params (role.id=061fff977e1f4d13a7b5e8e70edc3e92, scope.project.id=d844147ccee64e23b49af3bf86ac4c6f, user.id=ba2fc87b5e1a4d76976a702bdb6397e2) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:09.852558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44331d3-d50c-40f8-b50b-b2444dc546c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:09.854516 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44331d3-d50c-40f8-b50b-b2444dc546c6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:09.867946 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 159/318] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:08:09 2022] GET /identity/v3/role_assignments?user.id=ba2fc87b5e1a4d76976a702bdb6397e2&scope.project.id=d844147ccee64e23b49af3bf86ac4c6f&role.id=061fff977e1f4d13a7b5e8e70edc3e92 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:10.368118 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38c9b2-624b-41d5-ad9d-5c266a63eaff None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:10.368692 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38c9b2-624b-41d5-ad9d-5c266a63eaff None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:10.368692 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38c9b2-624b-41d5-ad9d-5c266a63eaff None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:10.370071 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 160/319] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:10.375411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4031897-1f23-4685-b35e-4b2187521723 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:10.375749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4031897-1f23-4685-b35e-4b2187521723 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:10.376029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4031897-1f23-4685-b35e-4b2187521723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:10.394563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e4031897-1f23-4685-b35e-4b2187521723 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:10.395789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e4031897-1f23-4685-b35e-4b2187521723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:10.462054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:10.472250 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 160/320] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:10.548358 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67805fe7-0064-4572-ba0f-a09b8c30897e None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:10.548631 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67805fe7-0064-4572-ba0f-a09b8c30897e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:10.548823 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67805fe7-0064-4572-ba0f-a09b8c30897e None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:10.550281 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 161/321] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:10.555152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926359f5-4121-4c81-8f0c-9d3b04bb9864 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:10.555441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926359f5-4121-4c81-8f0c-9d3b04bb9864 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:10.555740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926359f5-4121-4c81-8f0c-9d3b04bb9864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:10.574639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-926359f5-4121-4c81-8f0c-9d3b04bb9864 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:10.575851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-926359f5-4121-4c81-8f0c-9d3b04bb9864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:10.641616 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 161/322] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:10.740448 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67805fe7-0064-4572-ba0f-a09b8c30897e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:10.844355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67805fe7-0064-4572-ba0f-a09b8c30897e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:10.878300 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176aefd7-354d-4658-8aef-9c6348547cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:10.878963 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aefd7-354d-4658-8aef-9c6348547cab None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:10.879246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aefd7-354d-4658-8aef-9c6348547cab None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:10.879533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aefd7-354d-4658-8aef-9c6348547cab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:10.880819 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176aefd7-354d-4658-8aef-9c6348547cab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:10.882319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176aefd7-354d-4658-8aef-9c6348547cab None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:10.884242 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 162/323] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:10.890562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926359f5-4121-4c81-8f0c-9d3b04bb9864 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:10.912103 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176aefd7-354d-4658-8aef-9c6348547cab None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:10.976901 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:10.999695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926359f5-4121-4c81-8f0c-9d3b04bb9864 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:11.021073 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176aefd7-354d-4658-8aef-9c6348547cab None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:11.025702 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:11.035397 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22f599-700a-44cd-b719-551439a7976e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:11.036035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22f599-700a-44cd-b719-551439a7976e None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:11.036313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22f599-700a-44cd-b719-551439a7976e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:11.036584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22f599-700a-44cd-b719-551439a7976e None admin] PATH_INFO: `/v3/groups` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:11.037217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22f599-700a-44cd-b719-551439a7976e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:11.038667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22f599-700a-44cd-b719-551439a7976e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:11.053904 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 162/324] 10.222.0.33 () {60 vars in 1134 bytes} [Thu Oct 6 12:08:10 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:08:11.054713 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a19f5-6ad6-4cf4-8062-e639fe24a72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:11.055150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad6959-44f4-4da4-90bb-bb0b63c45884 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:11.055365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a19f5-6ad6-4cf4-8062-e639fe24a72e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:11.055460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad6959-44f4-4da4-90bb-bb0b63c45884 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:11.055667 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a19f5-6ad6-4cf4-8062-e639fe24a72e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:11.055752 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad6959-44f4-4da4-90bb-bb0b63c45884 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:11.055919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a19f5-6ad6-4cf4-8062-e639fe24a72e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:11.059763 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 163/325] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:11.063145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e2d6d-ed4e-41f3-a50e-9535b21d7014 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:11.063673 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e2d6d-ed4e-41f3-a50e-9535b21d7014 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:11.064088 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e2d6d-ed4e-41f3-a50e-9535b21d7014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:11.070182 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a19f5-6ad6-4cf4-8062-e639fe24a72e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:11.077234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-216a19f5-6ad6-4cf4-8062-e639fe24a72e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:08:11.077234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:11.077234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:11.077234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:11.077234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:11.077234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:11.077234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:11.077234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:11.077234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:11.077234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:11.077234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:11.077234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:11.077234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:11.077234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:11.077234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:11.077234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:11.078243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:11.078243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:11.078243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:11.078243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:11.078243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:11.078243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:11.078243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:11.078243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:11.078243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:11.078243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:11.078243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:11.078243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:11.078243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:11.078243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:11.078243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:11.078243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:11.079081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:11.079081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:11.079081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:11.079081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:11.079081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:11.079081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:11.079081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:11.079081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:11.079081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:11.079081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:11.079081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:08:11.079081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:11.079081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 163/326] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:08:11.084919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a19f5-6ad6-4cf4-8062-e639fe24a72e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:11.087184 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-5b6e2d6d-ed4e-41f3-a50e-9535b21d7014 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:11.088475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5b6e2d6d-ed4e-41f3-a50e-9535b21d7014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:11.163613 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 164/327] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:08:11.203731 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a19f5-6ad6-4cf4-8062-e639fe24a72e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:11.227757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:11.241605 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf8b55-e054-4f83-86e1-91888d2320ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:11.242221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf8b55-e054-4f83-86e1-91888d2320ed None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:11.242502 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf8b55-e054-4f83-86e1-91888d2320ed None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:11.242775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf8b55-e054-4f83-86e1-91888d2320ed None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:11.243562 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf8b55-e054-4f83-86e1-91888d2320ed None admin] RBAC: Adding query filter params (name=reader) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:11.243768 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf8b55-e054-4f83-86e1-91888d2320ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:11.245220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf8b55-e054-4f83-86e1-91888d2320ed None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:11.256099 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 164/328] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:08:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:11.262446 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e2d6d-ed4e-41f3-a50e-9535b21d7014 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:11.372464 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e2d6d-ed4e-41f3-a50e-9535b21d7014 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:11.390408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:11.411565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc020e8-4d01-47c5-9c83-52b9a1d91e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:11.412206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc020e8-4d01-47c5-9c83-52b9a1d91e00 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:11.412485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc020e8-4d01-47c5-9c83-52b9a1d91e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:11.412754 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc020e8-4d01-47c5-9c83-52b9a1d91e00 None admin] PATH_INFO: `/v3/users/03acf4a0d6b2488783e6071282f93c67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:11.413944 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc020e8-4d01-47c5-9c83-52b9a1d91e00 None admin] RBAC: Authorizing `identity:get_user(user_id=03acf4a0d6b2488783e6071282f93c67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:11.415325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc020e8-4d01-47c5-9c83-52b9a1d91e00 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:11.417411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 165/329] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:08:11 2022] GET /identity/v3/users/03acf4a0d6b2488783e6071282f93c67 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:11.423186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf8b55-e054-4f83-86e1-91888d2320ed None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:11.431124 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc020e8-4d01-47c5-9c83-52b9a1d91e00 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:11.557784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf8b55-e054-4f83-86e1-91888d2320ed None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:11.558669 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc020e8-4d01-47c5-9c83-52b9a1d91e00 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:11.597406 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ea9b7-3d35-4e2f-949c-fcf603a1da30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:11.598061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ea9b7-3d35-4e2f-949c-fcf603a1da30 None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:11.598338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ea9b7-3d35-4e2f-949c-fcf603a1da30 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:11.598609 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ea9b7-3d35-4e2f-949c-fcf603a1da30 None admin] PATH_INFO: `/v3/system/users/03acf4a0d6b2488783e6071282f93c67/roles/061fff977e1f4d13a7b5e8e70edc3e92` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:11.600060 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ea9b7-3d35-4e2f-949c-fcf603a1da30 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=03acf4a0d6b2488783e6071282f93c67, role_id=061fff977e1f4d13a7b5e8e70edc3e92)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:11.601416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ea9b7-3d35-4e2f-949c-fcf603a1da30 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:11.610951 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4d901-9383-427f-ba94-4500a1c6617f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:11.611645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4d901-9383-427f-ba94-4500a1c6617f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:11.612221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4d901-9383-427f-ba94-4500a1c6617f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:11.612346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4d901-9383-427f-ba94-4500a1c6617f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:11.614804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 165/330] 10.222.0.33 () {58 vars in 1334 bytes} [Thu Oct 6 12:08:11 2022] PUT /identity/v3/system/users/03acf4a0d6b2488783e6071282f93c67/roles/061fff977e1f4d13a7b5e8e70edc3e92 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:08:11.627898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4d901-9383-427f-ba94-4500a1c6617f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:11.633386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-2db4d901-9383-427f-ba94-4500a1c6617f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:11.633386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:11.633386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:11.633386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:11.633386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:11.633386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:11.633386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:11.633386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:11.633386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:11.633386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:11.633386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:11.633386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:11.633386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:11.633386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:11.633386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:11.633386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:11.634377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:11.634377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:11.634377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:11.634377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:11.634377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:11.634377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:11.634377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:11.634377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:11.634377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:11.634377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:11.634377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:11.634377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:11.634377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:11.634377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:11.634377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:11.634377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:11.635823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:11.635823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:11.635823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:11.635823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:11.635823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:11.635823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:11.635823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:11.635823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:11.635823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:11.635823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:11.635823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:11.635823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:11.635823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 166/331] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:08:11.642798 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ea9b7-3d35-4e2f-949c-fcf603a1da30 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:11.711963 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:11.777569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ea9b7-3d35-4e2f-949c-fcf603a1da30 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:11.816717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deeb062-6cb5-4560-87ce-5b40776edf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:11.817334 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deeb062-6cb5-4560-87ce-5b40776edf7e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:11.817612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deeb062-6cb5-4560-87ce-5b40776edf7e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:11.817885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deeb062-6cb5-4560-87ce-5b40776edf7e None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:11.818552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deeb062-6cb5-4560-87ce-5b40776edf7e None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:11.818852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deeb062-6cb5-4560-87ce-5b40776edf7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:11.820308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deeb062-6cb5-4560-87ce-5b40776edf7e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:11.832467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 166/332] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:08:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:11.838252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4d901-9383-427f-ba94-4500a1c6617f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:11.888287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:11.945490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4d901-9383-427f-ba94-4500a1c6617f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:11.981361 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1ffb7-4a66-4e71-aa49-886d82873032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:11.981913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ffb7-4a66-4e71-aa49-886d82873032 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:11.982191 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ffb7-4a66-4e71-aa49-886d82873032 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:11.982461 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ffb7-4a66-4e71-aa49-886d82873032 None admin] PATH_INFO: `/v3/users/576770327b8841eda5056fbe7e49579a` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:11.983711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1ffb7-4a66-4e71-aa49-886d82873032 None admin] RBAC: Authorizing `identity:get_user(user_id=576770327b8841eda5056fbe7e49579a)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:11.985151 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1ffb7-4a66-4e71-aa49-886d82873032 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:11.987049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 167/333] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:08:11 2022] GET /identity/v3/users/576770327b8841eda5056fbe7e49579a => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:11.992915 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deeb062-6cb5-4560-87ce-5b40776edf7e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:12.104982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deeb062-6cb5-4560-87ce-5b40776edf7e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:12.119902 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:12.147115 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0ce6d-9579-44c0-843f-1aae4caff08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:12.147758 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0ce6d-9579-44c0-843f-1aae4caff08b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:12.148048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0ce6d-9579-44c0-843f-1aae4caff08b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:12.148322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0ce6d-9579-44c0-843f-1aae4caff08b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:12.149050 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0ce6d-9579-44c0-843f-1aae4caff08b None admin] RBAC: Adding query filter params (role.id=bbea734b8a704d7ca8ec61bd45536cbe, user.id=576770327b8841eda5056fbe7e49579a, scope.system=all) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:12.149359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0ce6d-9579-44c0-843f-1aae4caff08b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:12.150858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0ce6d-9579-44c0-843f-1aae4caff08b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:12.163850 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 167/334] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Oct 6 12:08:11 2022] GET /identity/v3/role_assignments?user.id=576770327b8841eda5056fbe7e49579a&scope.system=all&role.id=bbea734b8a704d7ca8ec61bd45536cbe => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:13.882560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a533a80-9a8b-485f-bae7-97164820c57e None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:13.883583 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a533a80-9a8b-485f-bae7-97164820c57e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:13.884105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a533a80-9a8b-485f-bae7-97164820c57e None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:13.886149 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 168/335] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:13.891813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19712d25-82f3-489f-a9bf-b2f9a6749a80 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:13.892299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19712d25-82f3-489f-a9bf-b2f9a6749a80 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:13.892373 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19712d25-82f3-489f-a9bf-b2f9a6749a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:13.917275 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-19712d25-82f3-489f-a9bf-b2f9a6749a80 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:13.918492 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-19712d25-82f3-489f-a9bf-b2f9a6749a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:14.007799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123470c-e4b5-4e11-856e-0b67a703dffa None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:14.008391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 168/336] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:14.008648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123470c-e4b5-4e11-856e-0b67a703dffa None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:14.009172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123470c-e4b5-4e11-856e-0b67a703dffa None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:14.011000 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 169/337] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:14.033412 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23981cf8-6b4b-41f6-ba4a-c9d969a724e8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:14.033701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23981cf8-6b4b-41f6-ba4a-c9d969a724e8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:14.033969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23981cf8-6b4b-41f6-ba4a-c9d969a724e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:14.058373 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-23981cf8-6b4b-41f6-ba4a-c9d969a724e8 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:14.059611 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-23981cf8-6b4b-41f6-ba4a-c9d969a724e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:14.136509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 169/338] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:14.211892 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149aa73-ab34-46b0-ad2a-4e21d5420b74 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:14.212554 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149aa73-ab34-46b0-ad2a-4e21d5420b74 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:14.213578 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149aa73-ab34-46b0-ad2a-4e21d5420b74 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:14.215114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 170/339] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:14.220323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad41eb-e8a8-46cf-93d9-2a54c75608e9 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:14.220616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad41eb-e8a8-46cf-93d9-2a54c75608e9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:14.220892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad41eb-e8a8-46cf-93d9-2a54c75608e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:14.243461 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-12ad41eb-e8a8-46cf-93d9-2a54c75608e9 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:14.247542 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-12ad41eb-e8a8-46cf-93d9-2a54c75608e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:14.276135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:14.333646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 170/340] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:14.586655 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149aa73-ab34-46b0-ad2a-4e21d5420b74 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:14.630362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad41eb-e8a8-46cf-93d9-2a54c75608e9 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:14.785779 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149aa73-ab34-46b0-ad2a-4e21d5420b74 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:14.815298 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:14.820770 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad41eb-e8a8-46cf-93d9-2a54c75608e9 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:14.867703 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8b454-8d5d-4113-9df7-71d2406868b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:14.869398 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8b454-8d5d-4113-9df7-71d2406868b8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:14.869856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8b454-8d5d-4113-9df7-71d2406868b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:14.870543 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8b454-8d5d-4113-9df7-71d2406868b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:14.880729 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9729087-cf4d-4763-be71-cd7f2a53333f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:14.881908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9729087-cf4d-4763-be71-cd7f2a53333f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:14.883268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9729087-cf4d-4763-be71-cd7f2a53333f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:14.885199 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9729087-cf4d-4763-be71-cd7f2a53333f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:14.890803 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8b454-8d5d-4113-9df7-71d2406868b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:14.899777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-1ad8b454-8d5d-4113-9df7-71d2406868b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:08:14.899777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:14.899777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:14.899777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:14.899777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:14.899777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:14.899777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:14.899777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:14.899777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:14.899777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:14.899777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:14.899777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:14.899777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:14.899777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:14.899777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:14.899777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:14.901696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:14.901696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:14.901696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:14.901696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:14.901696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:14.901696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:14.901696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:14.901696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:14.901696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:14.901696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:14.901696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:14.901696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:14.901696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:14.901696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:14.901696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:14.901696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:14.902488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:14.902488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:14.902488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:14.902488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:14.902488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:14.902488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:14.902488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:14.902488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:14.902488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:14.902488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:14.902488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 12:08:14.902488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:14.905415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 171/341] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:14.906636 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8b454-8d5d-4113-9df7-71d2406868b8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:14.916400 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9729087-cf4d-4763-be71-cd7f2a53333f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:14.921339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-e9729087-cf4d-4763-be71-cd7f2a53333f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:14.921339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:14.921339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:14.921339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:14.921339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:14.921339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:14.921339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:14.921339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:14.921339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:14.921339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:14.921339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:14.921339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:14.921339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:14.921339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:14.921339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:14.921339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:14.922209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:14.922209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:14.922209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:14.922209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:14.922209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:14.922209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:14.922209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:14.922209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:14.922209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:14.922209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:14.922209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:14.922209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:14.922209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:14.922209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:14.922209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:14.922209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:14.923007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:14.923007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:14.923007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:14.923007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:14.923007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:14.923007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:14.923007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:14.923007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:14.923007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:14.923007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:14.923007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:14.923007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:14.923007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 171/342] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:14.923691 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9729087-cf4d-4763-be71-cd7f2a53333f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:14.953389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:15.084422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8b454-8d5d-4113-9df7-71d2406868b8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:15.129629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86dc87-4bac-4440-a9ae-a676d18815bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:15.130197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86dc87-4bac-4440-a9ae-a676d18815bb None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:15.130479 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86dc87-4bac-4440-a9ae-a676d18815bb None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:15.130753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86dc87-4bac-4440-a9ae-a676d18815bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:15.132718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9729087-cf4d-4763-be71-cd7f2a53333f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:15.142116 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86dc87-4bac-4440-a9ae-a676d18815bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:15.153135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-cd86dc87-4bac-4440-a9ae-a676d18815bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:08:15.153135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:15.153135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:15.153135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:15.153135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:15.153135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:15.153135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:15.153135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:15.153135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:15.153135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:15.153135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:15.153135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:15.153135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:15.153135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:15.153135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:15.153135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:15.154080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:15.154080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:15.154080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:15.154080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:15.154080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:15.154080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:15.154080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:15.154080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:15.154080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:15.154080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:15.154080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:15.154080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:15.154080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:15.154080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:15.154080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:15.154080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:15.154894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:15.154894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:15.154894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:15.154894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:15.154894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:15.154894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:15.154894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:15.154894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:15.154894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:15.154894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:15.154894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:08:15.154894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:15.154894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 172/343] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:08:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:15.157014 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86dc87-4bac-4440-a9ae-a676d18815bb None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:15.191337 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66f0eb-28e9-4a4b-b174-4ecb5bedf842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:15.191990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66f0eb-28e9-4a4b-b174-4ecb5bedf842 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:15.192273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66f0eb-28e9-4a4b-b174-4ecb5bedf842 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:15.192546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66f0eb-28e9-4a4b-b174-4ecb5bedf842 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:15.193215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66f0eb-28e9-4a4b-b174-4ecb5bedf842 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:15.193514 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66f0eb-28e9-4a4b-b174-4ecb5bedf842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:15.194905 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66f0eb-28e9-4a4b-b174-4ecb5bedf842 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:15.209988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 172/344] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:08:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:15.214010 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66f0eb-28e9-4a4b-b174-4ecb5bedf842 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:15.270670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:15.292828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:15.320228 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86dc87-4bac-4440-a9ae-a676d18815bb None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:15.364680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1c098-603a-4c79-81b6-e9a28c5a730c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:15.365575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1c098-603a-4c79-81b6-e9a28c5a730c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:15.365867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1c098-603a-4c79-81b6-e9a28c5a730c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:15.366151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1c098-603a-4c79-81b6-e9a28c5a730c None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:15.366836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1c098-603a-4c79-81b6-e9a28c5a730c None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:15.367151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1c098-603a-4c79-81b6-e9a28c5a730c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:15.367409 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66f0eb-28e9-4a4b-b174-4ecb5bedf842 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:15.368601 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1c098-603a-4c79-81b6-e9a28c5a730c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:15.379177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 173/345] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:08:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:15.380102 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1c098-603a-4c79-81b6-e9a28c5a730c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:15.410568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba2ce2-2c20-42d8-876b-07fcd9643d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:15.411530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba2ce2-2c20-42d8-876b-07fcd9643d6a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:15.412128 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba2ce2-2c20-42d8-876b-07fcd9643d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:15.412412 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba2ce2-2c20-42d8-876b-07fcd9643d6a None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:15.413082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba2ce2-2c20-42d8-876b-07fcd9643d6a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:15.413391 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba2ce2-2c20-42d8-876b-07fcd9643d6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:15.414777 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba2ce2-2c20-42d8-876b-07fcd9643d6a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:15.425431 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 173/346] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:15.426697 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7d950-22b3-41c1-84b5-47b9912a444a None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:15.426986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7d950-22b3-41c1-84b5-47b9912a444a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:15.427264 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7d950-22b3-41c1-84b5-47b9912a444a None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:15.428658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 174/347] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:15.429225 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7d950-22b3-41c1-84b5-47b9912a444a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:15.506760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1c098-603a-4c79-81b6-e9a28c5a730c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:15.552297 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2bf3a-6c52-41b8-ac6a-9d358f64f777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:15.552297 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2bf3a-6c52-41b8-ac6a-9d358f64f777 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:15.552297 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2bf3a-6c52-41b8-ac6a-9d358f64f777 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:15.552297 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2bf3a-6c52-41b8-ac6a-9d358f64f777 None admin] PATH_INFO: `/v3/users/03acf4a0d6b2488783e6071282f93c67` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:15.553394 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2bf3a-6c52-41b8-ac6a-9d358f64f777 None admin] RBAC: Authorizing `identity:get_user(user_id=03acf4a0d6b2488783e6071282f93c67)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:15.554849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2bf3a-6c52-41b8-ac6a-9d358f64f777 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:15.556841 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 174/348] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:08:15 2022] GET /identity/v3/users/03acf4a0d6b2488783e6071282f93c67 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:15.557439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2bf3a-6c52-41b8-ac6a-9d358f64f777 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:15.569666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:15.571400 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:15.585089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7d950-22b3-41c1-84b5-47b9912a444a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:15.632395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6fc68-da3e-413c-bc76-aea31b11c514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:15.632930 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6fc68-da3e-413c-bc76-aea31b11c514 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:15.633213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6fc68-da3e-413c-bc76-aea31b11c514 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:15.633487 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6fc68-da3e-413c-bc76-aea31b11c514 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:15.673962 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2bf3a-6c52-41b8-ac6a-9d358f64f777 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:15.713350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6fc68-da3e-413c-bc76-aea31b11c514 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:15.715135 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6fc68-da3e-413c-bc76-aea31b11c514 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:15.719686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970390c3-0e70-4f4d-9bb2-d3807409e770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:15.720310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970390c3-0e70-4f4d-9bb2-d3807409e770 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:15.720595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970390c3-0e70-4f4d-9bb2-d3807409e770 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:15.721383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970390c3-0e70-4f4d-9bb2-d3807409e770 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:15.818197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970390c3-0e70-4f4d-9bb2-d3807409e770 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:15.820096 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970390c3-0e70-4f4d-9bb2-d3807409e770 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:15.832177 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:15.834195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:15.844266 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 175/349] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:15.845332 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282b267-c994-488a-b3ef-a7702ede3a08 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:15.845609 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282b267-c994-488a-b3ef-a7702ede3a08 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:15.846789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282b267-c994-488a-b3ef-a7702ede3a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:15.868425 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c282b267-c994-488a-b3ef-a7702ede3a08 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:15.870549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c282b267-c994-488a-b3ef-a7702ede3a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:15.959700 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 175/350] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:15.960903 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d174260-cd8c-4872-b7f0-c4262dd507f2 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:15.961187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d174260-cd8c-4872-b7f0-c4262dd507f2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:15.961456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d174260-cd8c-4872-b7f0-c4262dd507f2 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:15.963491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 176/351] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:15.964274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d174260-cd8c-4872-b7f0-c4262dd507f2 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:15.966907 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 176/352] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:15.967793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282b267-c994-488a-b3ef-a7702ede3a08 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:16.091528 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d174260-cd8c-4872-b7f0-c4262dd507f2 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:16.101568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282b267-c994-488a-b3ef-a7702ede3a08 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:16.104197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:16.135832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:16.137391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f94440-5ff5-49a7-bb51-2166de9239c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:16.138354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f94440-5ff5-49a7-bb51-2166de9239c5 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:16.138911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f94440-5ff5-49a7-bb51-2166de9239c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:16.139405 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f94440-5ff5-49a7-bb51-2166de9239c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:16.141545 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f94440-5ff5-49a7-bb51-2166de9239c5 None admin] RBAC: Adding query filter params (role.id=061fff977e1f4d13a7b5e8e70edc3e92, user.id=03acf4a0d6b2488783e6071282f93c67, scope.system=all) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:16.142143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f94440-5ff5-49a7-bb51-2166de9239c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:16.143941 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f94440-5ff5-49a7-bb51-2166de9239c5 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:16.146371 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27891f4c-9678-4726-bcd3-e147fc3c74ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:16.147036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27891f4c-9678-4726-bcd3-e147fc3c74ea None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:16.147315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27891f4c-9678-4726-bcd3-e147fc3c74ea None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:16.147658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27891f4c-9678-4726-bcd3-e147fc3c74ea None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:16.149492 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27891f4c-9678-4726-bcd3-e147fc3c74ea None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:16.151908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27891f4c-9678-4726-bcd3-e147fc3c74ea None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:16.154270 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 177/353] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:15 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:16.155012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27891f4c-9678-4726-bcd3-e147fc3c74ea None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:16.163060 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 177/354] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Oct 6 12:08:15 2022] GET /identity/v3/role_assignments?user.id=03acf4a0d6b2488783e6071282f93c67&scope.system=all&role.id=061fff977e1f4d13a7b5e8e70edc3e92 => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:16.169390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3264a1b-4156-4726-8d96-753ffecf42b5 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:16.170013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3264a1b-4156-4726-8d96-753ffecf42b5 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:16.170513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3264a1b-4156-4726-8d96-753ffecf42b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:16.204822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d3264a1b-4156-4726-8d96-753ffecf42b5 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:16.206497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d3264a1b-4156-4726-8d96-753ffecf42b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:16.275546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27891f4c-9678-4726-bcd3-e147fc3c74ea None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:16.310093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 178/355] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:16.311480 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3264a1b-4156-4726-8d96-753ffecf42b5 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:16.326148 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1df22-f263-4ac6-b3bd-a8aeecffc47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:16.326869 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1df22-f263-4ac6-b3bd-a8aeecffc47c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:16.327632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1df22-f263-4ac6-b3bd-a8aeecffc47c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:16.327981 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1df22-f263-4ac6-b3bd-a8aeecffc47c None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:16.329617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1df22-f263-4ac6-b3bd-a8aeecffc47c None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:16.331420 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1df22-f263-4ac6-b3bd-a8aeecffc47c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:16.334362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 178/356] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:16 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:16.335550 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d132d-ed54-4129-a1db-6e1e231b6455 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:16.335839 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d132d-ed54-4129-a1db-6e1e231b6455 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:16.336386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d132d-ed54-4129-a1db-6e1e231b6455 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:16.342425 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 179/357] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:16.342425 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d132d-ed54-4129-a1db-6e1e231b6455 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:16.424136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:16.443256 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:16.453439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3264a1b-4156-4726-8d96-753ffecf42b5 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:16.469639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d132d-ed54-4129-a1db-6e1e231b6455 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:16.498882 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bd21d-2c2b-4edc-bf16-1e1d7541d604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:16.499817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bd21d-2c2b-4edc-bf16-1e1d7541d604 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:16.500227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bd21d-2c2b-4edc-bf16-1e1d7541d604 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:16.500646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bd21d-2c2b-4edc-bf16-1e1d7541d604 None admin] PATH_INFO: `/v3/groups/1f265fd53f4a45c387c0b90fafb42061` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:16.508118 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bd21d-2c2b-4edc-bf16-1e1d7541d604 None admin] RBAC: Authorizing `identity:get_group(group_id=1f265fd53f4a45c387c0b90fafb42061)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:16.510543 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bd21d-2c2b-4edc-bf16-1e1d7541d604 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:16.512821 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 179/358] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Oct 6 12:08:16 2022] GET /identity/v3/groups/1f265fd53f4a45c387c0b90fafb42061 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:16.513613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bd21d-2c2b-4edc-bf16-1e1d7541d604 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:16.516489 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2d687-eb29-4be2-a5f4-291428422c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:16.517127 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2d687-eb29-4be2-a5f4-291428422c6e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:16.517399 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2d687-eb29-4be2-a5f4-291428422c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:16.517658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2d687-eb29-4be2-a5f4-291428422c6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:16.534871 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2d687-eb29-4be2-a5f4-291428422c6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:16.540215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-9af2d687-eb29-4be2-a5f4-291428422c6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:16.540215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:16.540215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:16.540215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:16.540215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:16.540215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:16.540215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:16.540215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:16.540215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:16.540215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:16.540215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:16.540215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:16.540215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:16.540215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:16.540215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:16.540215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:16.541653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:16.541653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:16.541653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:16.541653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:16.541653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:16.541653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:16.541653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:16.541653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:16.541653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:16.541653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:16.541653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:16.541653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:16.541653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:16.541653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:16.541653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:16.541653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:16.542485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:16.542485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:16.542485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:16.542485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:16.542485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:16.542485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:16.542485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:16.542485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:16.542485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:16.542485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:16.542485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:16.542485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:16.542485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 180/359] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:16.543152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b2ae2-e6f3-4cb1-bcb2-34e919c8b429 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:16.543270 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b2ae2-e6f3-4cb1-bcb2-34e919c8b429 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:16.547593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b2ae2-e6f3-4cb1-bcb2-34e919c8b429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:16.573455 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-0b2b2ae2-e6f3-4cb1-bcb2-34e919c8b429 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:16.577542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0b2b2ae2-e6f3-4cb1-bcb2-34e919c8b429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:16.653271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bd21d-2c2b-4edc-bf16-1e1d7541d604 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:16.654587 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 181/360] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:16.655862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b2ae2-e6f3-4cb1-bcb2-34e919c8b429 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:16.697064 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95648acf-cc1c-4725-b8fa-f4b279d5feb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:16.697955 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95648acf-cc1c-4725-b8fa-f4b279d5feb8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:16.698247 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95648acf-cc1c-4725-b8fa-f4b279d5feb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:16.698523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95648acf-cc1c-4725-b8fa-f4b279d5feb8 None admin] PATH_INFO: `/v3/groups/1f265fd53f4a45c387c0b90fafb42061` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:16.701216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95648acf-cc1c-4725-b8fa-f4b279d5feb8 None admin] RBAC: Authorizing `identity:get_group(group_id=1f265fd53f4a45c387c0b90fafb42061)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:16.701216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95648acf-cc1c-4725-b8fa-f4b279d5feb8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:16.703121 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 180/361] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Oct 6 12:08:16 2022] GET /identity/v3/groups/1f265fd53f4a45c387c0b90fafb42061 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:16.707537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95648acf-cc1c-4725-b8fa-f4b279d5feb8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:16.726529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:16.741827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:16.786381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b2ae2-e6f3-4cb1-bcb2-34e919c8b429 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:16.824183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa029f-7cab-455a-9835-f5c06f47f32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:16.824798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa029f-7cab-455a-9835-f5c06f47f32f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:16.825082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa029f-7cab-455a-9835-f5c06f47f32f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:16.825356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa029f-7cab-455a-9835-f5c06f47f32f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:16.826089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa029f-7cab-455a-9835-f5c06f47f32f None admin] RBAC: Adding query filter params (group.id=1f265fd53f4a45c387c0b90fafb42061, role.id=bbea734b8a704d7ca8ec61bd45536cbe, scope.project.id=b427802e13664086974c8d53f5bfbc88) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:16.826394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa029f-7cab-455a-9835-f5c06f47f32f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:16.828007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa029f-7cab-455a-9835-f5c06f47f32f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:16.834121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 182/362] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Oct 6 12:08:16 2022] GET /identity/v3/role_assignments?group.id=1f265fd53f4a45c387c0b90fafb42061&scope.project.id=b427802e13664086974c8d53f5bfbc88&role.id=bbea734b8a704d7ca8ec61bd45536cbe => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:16.834921 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa029f-7cab-455a-9835-f5c06f47f32f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:16.837126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95648acf-cc1c-4725-b8fa-f4b279d5feb8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:16.878472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84683e7-2ce4-43ee-8d0a-e25da6ae416f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:16.879660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84683e7-2ce4-43ee-8d0a-e25da6ae416f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:16.880147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84683e7-2ce4-43ee-8d0a-e25da6ae416f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:16.880994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84683e7-2ce4-43ee-8d0a-e25da6ae416f None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:16.881927 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84683e7-2ce4-43ee-8d0a-e25da6ae416f None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:16.882443 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84683e7-2ce4-43ee-8d0a-e25da6ae416f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:16.884864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84683e7-2ce4-43ee-8d0a-e25da6ae416f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:16.897310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 181/363] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:08:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:16.899080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84683e7-2ce4-43ee-8d0a-e25da6ae416f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:17.000998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa029f-7cab-455a-9835-f5c06f47f32f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:17.022390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:17.026075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:17.034408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84683e7-2ce4-43ee-8d0a-e25da6ae416f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:17.055642 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4b7ac-bb3f-47d8-a4b0-3318d5c1cb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:17.056585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4b7ac-bb3f-47d8-a4b0-3318d5c1cb2f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:17.057547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4b7ac-bb3f-47d8-a4b0-3318d5c1cb2f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:17.057547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4b7ac-bb3f-47d8-a4b0-3318d5c1cb2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:17.070965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4b7ac-bb3f-47d8-a4b0-3318d5c1cb2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:17.075906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3191c7-eaa2-4c45-adf4-70383c1c5fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:17.076524 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3191c7-eaa2-4c45-adf4-70383c1c5fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:17.076799 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3191c7-eaa2-4c45-adf4-70383c1c5fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:17.077003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-48d4b7ac-bb3f-47d8-a4b0-3318d5c1cb2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:08:17.077003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:17.077003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:17.077003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:17.077003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:17.077003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:17.077003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:17.077003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:17.077003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:17.077003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:17.077003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:17.077003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:17.077003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:17.077003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:17.077003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:17.077003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:17.077868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:17.077868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:17.077868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:17.077868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:17.077868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:17.077868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:17.077868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:17.077868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:17.077868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:17.077868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:17.077868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:17.077868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:17.077868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:17.077868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:17.077868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:17.077868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:17.078696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:17.078696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:17.078696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:17.078696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:17.078696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:17.078696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:17.078696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:17.078696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:17.078696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:17.078696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:17.078696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:08:17.078696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:17.079318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3191c7-eaa2-4c45-adf4-70383c1c5fd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:17.079318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3191c7-eaa2-4c45-adf4-70383c1c5fd0 None admin] RBAC: Adding query filter params (group.id=1f265fd53f4a45c387c0b90fafb42061, role.id=2c11a95c1ecf4988a95aad8d96ea56e9, scope.project.id=b427802e13664086974c8d53f5bfbc88) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:17.079318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3191c7-eaa2-4c45-adf4-70383c1c5fd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:17.080083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 183/364] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:08:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:17.080083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4b7ac-bb3f-47d8-a4b0-3318d5c1cb2f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:17.080252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3191c7-eaa2-4c45-adf4-70383c1c5fd0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:17.086266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 182/365] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Oct 6 12:08:16 2022] GET /identity/v3/role_assignments?group.id=1f265fd53f4a45c387c0b90fafb42061&scope.project.id=b427802e13664086974c8d53f5bfbc88&role.id=2c11a95c1ecf4988a95aad8d96ea56e9 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:17.087076 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3191c7-eaa2-4c45-adf4-70383c1c5fd0 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:17.208671 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4b7ac-bb3f-47d8-a4b0-3318d5c1cb2f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:17.219680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3191c7-eaa2-4c45-adf4-70383c1c5fd0 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:17.260529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1ec43-5462-4054-874d-fda379fd9ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:17.261158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1ec43-5462-4054-874d-fda379fd9ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:17.261436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1ec43-5462-4054-874d-fda379fd9ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:17.261710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1ec43-5462-4054-874d-fda379fd9ee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:17.295744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283876c9-25d8-42a1-bc2f-5594c77a753d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:17.296658 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283876c9-25d8-42a1-bc2f-5594c77a753d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:17.296942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283876c9-25d8-42a1-bc2f-5594c77a753d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:17.297212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283876c9-25d8-42a1-bc2f-5594c77a753d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:17.313104 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283876c9-25d8-42a1-bc2f-5594c77a753d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:17.319541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-283876c9-25d8-42a1-bc2f-5594c77a753d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:08:17.319541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:17.319541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:17.319541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:17.319541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:17.319541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:17.319541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:17.319541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:17.319541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:17.319541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:17.319541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:17.319541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:17.319541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:17.319541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:17.319541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:17.319541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:17.320453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:17.320453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:17.320453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:17.320453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:17.320453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:17.320453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:17.320453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:17.320453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:17.320453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:17.320453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:17.320453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:17.320453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:17.320453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:17.320453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:17.320453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:17.320453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:17.321274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:17.321274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:17.321274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:17.321274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:17.321274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:17.321274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:17.321274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:17.321274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:17.321274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:17.321274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:17.321274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:08:17.321274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:17.325613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 183/366] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:08:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:17.326480 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283876c9-25d8-42a1-bc2f-5594c77a753d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:17.327255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:17.356661 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:17.360199 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1ec43-5462-4054-874d-fda379fd9ee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:17.361679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1ec43-5462-4054-874d-fda379fd9ee4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:17.470299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283876c9-25d8-42a1-bc2f-5594c77a753d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:17.489562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 184/367] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:17.490770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1ec43-5462-4054-874d-fda379fd9ee4 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:17.515445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0306c-b77f-4ae8-be9c-f3d5877c008a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:17.516464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0306c-b77f-4ae8-be9c-f3d5877c008a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:17.516945 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0306c-b77f-4ae8-be9c-f3d5877c008a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:17.517457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0306c-b77f-4ae8-be9c-f3d5877c008a None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:17.518445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0306c-b77f-4ae8-be9c-f3d5877c008a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:17.519102 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0306c-b77f-4ae8-be9c-f3d5877c008a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:17.521618 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0306c-b77f-4ae8-be9c-f3d5877c008a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:17.532006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 184/368] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:17.533112 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0306c-b77f-4ae8-be9c-f3d5877c008a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:17.575760 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:17.602225 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1ec43-5462-4054-874d-fda379fd9ee4 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:17.623672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:17.641860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b545dc-39ac-4942-ab3d-ba16e6266e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:17.642473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b545dc-39ac-4942-ab3d-ba16e6266e21 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:17.642748 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b545dc-39ac-4942-ab3d-ba16e6266e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:17.643015 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b545dc-39ac-4942-ab3d-ba16e6266e21 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:17.643768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b545dc-39ac-4942-ab3d-ba16e6266e21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:17.643995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b545dc-39ac-4942-ab3d-ba16e6266e21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:17.645375 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b545dc-39ac-4942-ab3d-ba16e6266e21 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:17.655143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0306c-b77f-4ae8-be9c-f3d5877c008a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:17.655306 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 185/369] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:17.659988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b545dc-39ac-4942-ab3d-ba16e6266e21 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:17.706075 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417db9e-8dda-421b-9cb5-73517f9d1d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:17.706722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417db9e-8dda-421b-9cb5-73517f9d1d2b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:17.707001 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417db9e-8dda-421b-9cb5-73517f9d1d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:17.707267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417db9e-8dda-421b-9cb5-73517f9d1d2b None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:17.708417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417db9e-8dda-421b-9cb5-73517f9d1d2b None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:17.709887 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417db9e-8dda-421b-9cb5-73517f9d1d2b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:17.712103 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 185/370] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:17 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:17.713381 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417db9e-8dda-421b-9cb5-73517f9d1d2b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:17.784183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b545dc-39ac-4942-ab3d-ba16e6266e21 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:17.827422 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48981ee1-2956-4ea9-ac00-78cb7839f348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:17.828711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48981ee1-2956-4ea9-ac00-78cb7839f348 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:17.829666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48981ee1-2956-4ea9-ac00-78cb7839f348 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:17.830146 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48981ee1-2956-4ea9-ac00-78cb7839f348 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:17.840888 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:17.841340 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417db9e-8dda-421b-9cb5-73517f9d1d2b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:17.892532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c04ce1-3994-4296-bbc5-dab45d6021e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:17.892532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c04ce1-3994-4296-bbc5-dab45d6021e0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:17.892532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c04ce1-3994-4296-bbc5-dab45d6021e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:17.892881 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c04ce1-3994-4296-bbc5-dab45d6021e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:17.913598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:17.926075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48981ee1-2956-4ea9-ac00-78cb7839f348 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:17.927627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48981ee1-2956-4ea9-ac00-78cb7839f348 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:17.977027 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c04ce1-3994-4296-bbc5-dab45d6021e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:17.981490 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c04ce1-3994-4296-bbc5-dab45d6021e0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:18.076357 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 186/371] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:18.077893 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48981ee1-2956-4ea9-ac00-78cb7839f348 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:18.125258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 186/372] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:18.126687 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c04ce1-3994-4296-bbc5-dab45d6021e0 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:18.142402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:18.184472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:18.226113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48981ee1-2956-4ea9-ac00-78cb7839f348 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:18.262313 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c04ce1-3994-4296-bbc5-dab45d6021e0 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:18.280973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861db6c-118d-4e36-b86d-e54d950dc42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:18.282312 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861db6c-118d-4e36-b86d-e54d950dc42c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:18.282789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861db6c-118d-4e36-b86d-e54d950dc42c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:18.283251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861db6c-118d-4e36-b86d-e54d950dc42c None admin] PATH_INFO: `/v3/groups/1f265fd53f4a45c387c0b90fafb42061` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:18.285165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861db6c-118d-4e36-b86d-e54d950dc42c None admin] RBAC: Authorizing `identity:get_group(group_id=1f265fd53f4a45c387c0b90fafb42061)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:18.287346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861db6c-118d-4e36-b86d-e54d950dc42c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:18.290081 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 187/373] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Oct 6 12:08:18 2022] GET /identity/v3/groups/1f265fd53f4a45c387c0b90fafb42061 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:18.291902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861db6c-118d-4e36-b86d-e54d950dc42c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:18.301078 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98127f5-abf3-42ab-afb7-d224dba9ffb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:18.301721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98127f5-abf3-42ab-afb7-d224dba9ffb2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:18.302004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98127f5-abf3-42ab-afb7-d224dba9ffb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:18.302277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98127f5-abf3-42ab-afb7-d224dba9ffb2 None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:18.303615 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98127f5-abf3-42ab-afb7-d224dba9ffb2 None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:18.304886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98127f5-abf3-42ab-afb7-d224dba9ffb2 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:18.307340 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 187/374] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:18 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:18.308374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98127f5-abf3-42ab-afb7-d224dba9ffb2 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:18.416769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861db6c-118d-4e36-b86d-e54d950dc42c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:18.424617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:18.436573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98127f5-abf3-42ab-afb7-d224dba9ffb2 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:18.451944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:18.457406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38702b44-e231-43ba-bb44-703691c271e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:18.458150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38702b44-e231-43ba-bb44-703691c271e0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:18.458592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38702b44-e231-43ba-bb44-703691c271e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:18.458829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38702b44-e231-43ba-bb44-703691c271e0 None admin] PATH_INFO: `/v3/projects/5568e23a42b74b0a8d5f679d197b7521` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:18.460519 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38702b44-e231-43ba-bb44-703691c271e0 None admin] RBAC: Authorizing `identity:get_project(project_id=5568e23a42b74b0a8d5f679d197b7521)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:18.461615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38702b44-e231-43ba-bb44-703691c271e0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:18.469266 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 188/375] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:18 2022] GET /identity/v3/projects/5568e23a42b74b0a8d5f679d197b7521 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:18.469266 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38702b44-e231-43ba-bb44-703691c271e0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:18.484932 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9d9ff-b9c7-4a9b-bc03-0b0ea31a31f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:18.485774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9d9ff-b9c7-4a9b-bc03-0b0ea31a31f8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:18.486276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9d9ff-b9c7-4a9b-bc03-0b0ea31a31f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:18.486738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9d9ff-b9c7-4a9b-bc03-0b0ea31a31f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:18.487680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9d9ff-b9c7-4a9b-bc03-0b0ea31a31f8 None admin] RBAC: Adding query filter params (group.id=1f265fd53f4a45c387c0b90fafb42061, role.id=bbea734b8a704d7ca8ec61bd45536cbe, scope.project.id=d844147ccee64e23b49af3bf86ac4c6f) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:18.488183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9d9ff-b9c7-4a9b-bc03-0b0ea31a31f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:18.490203 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9d9ff-b9c7-4a9b-bc03-0b0ea31a31f8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:18.496723 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 188/376] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Oct 6 12:08:18 2022] GET /identity/v3/role_assignments?group.id=1f265fd53f4a45c387c0b90fafb42061&scope.project.id=d844147ccee64e23b49af3bf86ac4c6f&role.id=bbea734b8a704d7ca8ec61bd45536cbe => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:18.497878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9d9ff-b9c7-4a9b-bc03-0b0ea31a31f8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:18.613725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38702b44-e231-43ba-bb44-703691c271e0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:18.617956 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9d9ff-b9c7-4a9b-bc03-0b0ea31a31f8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:18.660835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4db9d0-6566-44a5-b2d0-8cc6953158bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:18.661832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4db9d0-6566-44a5-b2d0-8cc6953158bc None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:18.662303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4db9d0-6566-44a5-b2d0-8cc6953158bc None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:18.662773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4db9d0-6566-44a5-b2d0-8cc6953158bc None admin] PATH_INFO: `/v3/groups/be0d12feeaec4b4cb3faf295204a3026` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:18.669723 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4db9d0-6566-44a5-b2d0-8cc6953158bc None admin] RBAC: Authorizing `identity:get_group(group_id=be0d12feeaec4b4cb3faf295204a3026)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:18.671563 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4db9d0-6566-44a5-b2d0-8cc6953158bc None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:18.673793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 189/377] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Oct 6 12:08:18 2022] GET /identity/v3/groups/be0d12feeaec4b4cb3faf295204a3026 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:18.681111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4db9d0-6566-44a5-b2d0-8cc6953158bc None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:18.719458 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf7d8e-f5f4-4ad6-b5c8-548d11504b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:18.720183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf7d8e-f5f4-4ad6-b5c8-548d11504b45 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:18.720458 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf7d8e-f5f4-4ad6-b5c8-548d11504b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:18.720726 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf7d8e-f5f4-4ad6-b5c8-548d11504b45 None admin] PATH_INFO: `/v3/groups/1f265fd53f4a45c387c0b90fafb42061` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:18.721954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf7d8e-f5f4-4ad6-b5c8-548d11504b45 None admin] RBAC: Authorizing `identity:get_group(group_id=1f265fd53f4a45c387c0b90fafb42061)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:18.723369 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf7d8e-f5f4-4ad6-b5c8-548d11504b45 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:18.725219 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 189/378] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Oct 6 12:08:18 2022] GET /identity/v3/groups/1f265fd53f4a45c387c0b90fafb42061 => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:18.730986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf7d8e-f5f4-4ad6-b5c8-548d11504b45 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:18.734913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:18.799611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:18.846787 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4db9d0-6566-44a5-b2d0-8cc6953158bc None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:18.894752 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf7d8e-f5f4-4ad6-b5c8-548d11504b45 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:18.914780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f03533-f9ef-4df3-9e69-093911b9f677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:18.915403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f03533-f9ef-4df3-9e69-093911b9f677 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:18.915738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f03533-f9ef-4df3-9e69-093911b9f677 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:18.916016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f03533-f9ef-4df3-9e69-093911b9f677 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:18.918353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f03533-f9ef-4df3-9e69-093911b9f677 None admin] RBAC: Adding query filter params (group.id=be0d12feeaec4b4cb3faf295204a3026, role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=5568e23a42b74b0a8d5f679d197b7521) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:18.918353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f03533-f9ef-4df3-9e69-093911b9f677 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:18.918610 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f03533-f9ef-4df3-9e69-093911b9f677 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:18.929713 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 190/379] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Oct 6 12:08:18 2022] GET /identity/v3/role_assignments?group.id=be0d12feeaec4b4cb3faf295204a3026&scope.project.id=5568e23a42b74b0a8d5f679d197b7521&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:18.951661 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1ecc2-f0ba-4910-8476-79837cbc6af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:18.952661 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1ecc2-f0ba-4910-8476-79837cbc6af5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:18.953200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1ecc2-f0ba-4910-8476-79837cbc6af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:18.953722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1ecc2-f0ba-4910-8476-79837cbc6af5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:18.954642 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1ecc2-f0ba-4910-8476-79837cbc6af5 None admin] RBAC: Adding query filter params (group.id=1f265fd53f4a45c387c0b90fafb42061, role.id=2c11a95c1ecf4988a95aad8d96ea56e9, scope.project.id=d844147ccee64e23b49af3bf86ac4c6f) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:18.955162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1ecc2-f0ba-4910-8476-79837cbc6af5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:18.956878 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1ecc2-f0ba-4910-8476-79837cbc6af5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:18.968545 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 190/380] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Oct 6 12:08:18 2022] GET /identity/v3/role_assignments?group.id=1f265fd53f4a45c387c0b90fafb42061&scope.project.id=d844147ccee64e23b49af3bf86ac4c6f&role.id=2c11a95c1ecf4988a95aad8d96ea56e9 => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:19.362858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b4fb8-2a38-43f1-afae-ead85873554a None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:19.363359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b4fb8-2a38-43f1-afae-ead85873554a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:19.363433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b4fb8-2a38-43f1-afae-ead85873554a None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:19.365062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 191/381] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:19.371065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ae88b-2302-4a56-8a12-bc394adfc1ac None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:19.371532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ae88b-2302-4a56-8a12-bc394adfc1ac None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:19.371717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ae88b-2302-4a56-8a12-bc394adfc1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:19.396519 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8c2ae88b-2302-4a56-8a12-bc394adfc1ac None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:19.397735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8c2ae88b-2302-4a56-8a12-bc394adfc1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:19.486094 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 191/382] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:08:19.676394 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087166a-8050-4ebf-a603-0ba73f18da55 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:19.676723 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087166a-8050-4ebf-a603-0ba73f18da55 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:19.676954 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087166a-8050-4ebf-a603-0ba73f18da55 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:19.678426 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 192/383] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:19.687091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09b4c9-7ea4-44f4-aebb-6175cd15221c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:19.687091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09b4c9-7ea4-44f4-aebb-6175cd15221c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:19.687091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09b4c9-7ea4-44f4-aebb-6175cd15221c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:19.703830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6f09b4c9-7ea4-44f4-aebb-6175cd15221c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:19.705333 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6f09b4c9-7ea4-44f4-aebb-6175cd15221c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:19.735619 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:19.777772 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 192/384] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:20.046807 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087166a-8050-4ebf-a603-0ba73f18da55 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:20.068060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09b4c9-7ea4-44f4-aebb-6175cd15221c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:20.209404 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087166a-8050-4ebf-a603-0ba73f18da55 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:20.211669 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09b4c9-7ea4-44f4-aebb-6175cd15221c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:20.212773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:20.248274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f223c-a2d1-4e97-831a-379274344427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:20.249108 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f223c-a2d1-4e97-831a-379274344427 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:20.249517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f223c-a2d1-4e97-831a-379274344427 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:20.250001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f223c-a2d1-4e97-831a-379274344427 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:20.262085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f223c-a2d1-4e97-831a-379274344427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:20.267214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-851f223c-a2d1-4e97-831a-379274344427 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:08:20.267214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:20.267214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:20.267214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:20.267214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:20.267214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:20.267214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:20.267214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:20.267214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:20.267214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:20.267214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:20.267214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:20.267214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:20.267214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:20.267214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:20.267214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:20.268299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:20.268299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:20.268299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:20.268299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:20.268299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:20.268299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:20.268299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:20.268299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:20.268299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:20.268299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:20.268299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:20.268299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:20.268299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:20.268299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:20.268299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:20.268299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:20.269283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:20.269283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:20.269283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:20.269283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:20.269283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:20.269283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:20.269283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:20.269283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:20.269283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:20.269283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:20.269283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:08:20.269283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:20.271467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 193/385] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:08:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:20.277388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f223c-a2d1-4e97-831a-379274344427 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:20.297460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68753c1-025f-4821-b2bb-be98c81eec82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:20.298023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68753c1-025f-4821-b2bb-be98c81eec82 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:20.299581 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68753c1-025f-4821-b2bb-be98c81eec82 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:20.299865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68753c1-025f-4821-b2bb-be98c81eec82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:20.323832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68753c1-025f-4821-b2bb-be98c81eec82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:20.329646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-b68753c1-025f-4821-b2bb-be98c81eec82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:20.329646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:20.329646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:20.329646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:20.329646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:20.329646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:20.329646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:20.329646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:20.329646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:20.329646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:20.329646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:20.329646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:20.329646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:20.329646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:20.329646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:20.329646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:20.330533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:20.330533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:20.330533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:20.330533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:20.330533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:20.330533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:20.330533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:20.330533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:20.330533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:20.330533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:20.330533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:20.330533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:20.330533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:20.330533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:20.330533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:20.330533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:20.331323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:20.331323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:20.331323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:20.331323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:20.331323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:20.331323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:20.331323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:20.331323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:20.331323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:20.331323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:20.331323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:20.331323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:20.333342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:20.338392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 193/386] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:20.344744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68753c1-025f-4821-b2bb-be98c81eec82 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:20.420818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f223c-a2d1-4e97-831a-379274344427 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:20.469646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463451f-b6d6-4349-b3e1-eff87045077d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:20.470271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463451f-b6d6-4349-b3e1-eff87045077d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:20.470552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463451f-b6d6-4349-b3e1-eff87045077d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:20.470825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463451f-b6d6-4349-b3e1-eff87045077d None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:20.471491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463451f-b6d6-4349-b3e1-eff87045077d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:20.471823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463451f-b6d6-4349-b3e1-eff87045077d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:20.473210 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463451f-b6d6-4349-b3e1-eff87045077d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:20.482838 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 194/387] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:20.489566 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463451f-b6d6-4349-b3e1-eff87045077d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:20.552320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68753c1-025f-4821-b2bb-be98c81eec82 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:20.622831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812732f-8305-4597-a699-6359aa093dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:20.623453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812732f-8305-4597-a699-6359aa093dfe None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:20.623757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812732f-8305-4597-a699-6359aa093dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:20.624032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812732f-8305-4597-a699-6359aa093dfe None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:20.624696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812732f-8305-4597-a699-6359aa093dfe None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:20.624995 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812732f-8305-4597-a699-6359aa093dfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:20.626365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812732f-8305-4597-a699-6359aa093dfe None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:20.641551 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 194/388] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:08:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:20.647207 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812732f-8305-4597-a699-6359aa093dfe None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:20.656220 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:20.686347 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463451f-b6d6-4349-b3e1-eff87045077d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:20.699684 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:20.755890 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20c526-96d5-423e-8a63-4924c2a91d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:20.756520 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20c526-96d5-423e-8a63-4924c2a91d78 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:20.756798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20c526-96d5-423e-8a63-4924c2a91d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:20.757067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20c526-96d5-423e-8a63-4924c2a91d78 None admin] PATH_INFO: `/v3/groups/1f265fd53f4a45c387c0b90fafb42061` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:20.758732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20c526-96d5-423e-8a63-4924c2a91d78 None admin] RBAC: Authorizing `identity:get_group(group_id=1f265fd53f4a45c387c0b90fafb42061)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:20.760246 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20c526-96d5-423e-8a63-4924c2a91d78 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:20.762120 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 195/389] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Oct 6 12:08:20 2022] GET /identity/v3/groups/1f265fd53f4a45c387c0b90fafb42061 => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:20.767301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20c526-96d5-423e-8a63-4924c2a91d78 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:20.826882 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812732f-8305-4597-a699-6359aa093dfe None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:20.888205 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1546b-78db-490f-a399-02d67ea868c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:20.889021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1546b-78db-490f-a399-02d67ea868c5 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:20.889309 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1546b-78db-490f-a399-02d67ea868c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:20.889584 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1546b-78db-490f-a399-02d67ea868c5 None admin] PATH_INFO: `/v3/groups/1f265fd53f4a45c387c0b90fafb42061` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:20.890830 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1546b-78db-490f-a399-02d67ea868c5 None admin] RBAC: Authorizing `identity:get_group(group_id=1f265fd53f4a45c387c0b90fafb42061)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:20.892310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1546b-78db-490f-a399-02d67ea868c5 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:20.898441 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 195/390] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Oct 6 12:08:20 2022] GET /identity/v3/groups/1f265fd53f4a45c387c0b90fafb42061 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:20.899806 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1546b-78db-490f-a399-02d67ea868c5 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:20.940134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20c526-96d5-423e-8a63-4924c2a91d78 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:20.985619 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2d0bd-287a-401c-b4b8-97c53623ed9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:20.986266 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2d0bd-287a-401c-b4b8-97c53623ed9e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:20.986550 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2d0bd-287a-401c-b4b8-97c53623ed9e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:20.986826 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2d0bd-287a-401c-b4b8-97c53623ed9e None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:20.992241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2d0bd-287a-401c-b4b8-97c53623ed9e None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:20.993803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2d0bd-287a-401c-b4b8-97c53623ed9e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:20.995844 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 196/391] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:20 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:21.001381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2d0bd-287a-401c-b4b8-97c53623ed9e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:21.027885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:21.039350 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1546b-78db-490f-a399-02d67ea868c5 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:21.054625 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:21.117588 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bac5d0-4800-43ff-bc2d-c48b030cf21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:21.118250 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bac5d0-4800-43ff-bc2d-c48b030cf21e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:21.118610 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bac5d0-4800-43ff-bc2d-c48b030cf21e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:21.118889 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bac5d0-4800-43ff-bc2d-c48b030cf21e None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:21.121123 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bac5d0-4800-43ff-bc2d-c48b030cf21e None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:21.122581 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bac5d0-4800-43ff-bc2d-c48b030cf21e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:21.128926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 196/392] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:20 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:21.136434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bac5d0-4800-43ff-bc2d-c48b030cf21e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:21.151073 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2d0bd-287a-401c-b4b8-97c53623ed9e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:21.205694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fb53b-33f9-42ab-b663-897c026867d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:21.206354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fb53b-33f9-42ab-b663-897c026867d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:21.206637 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fb53b-33f9-42ab-b663-897c026867d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:21.206913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fb53b-33f9-42ab-b663-897c026867d3 None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88/groups/1f265fd53f4a45c387c0b90fafb42061/roles/2c11a95c1ecf4988a95aad8d96ea56e9` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:21.209172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fb53b-33f9-42ab-b663-897c026867d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b427802e13664086974c8d53f5bfbc88, group_id=1f265fd53f4a45c387c0b90fafb42061, role_id=2c11a95c1ecf4988a95aad8d96ea56e9)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:21.211271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fb53b-33f9-42ab-b663-897c026867d3 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:21.229282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 197/393] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Oct 6 12:08:21 2022] PUT /identity/v3/projects/b427802e13664086974c8d53f5bfbc88/groups/1f265fd53f4a45c387c0b90fafb42061/roles/2c11a95c1ecf4988a95aad8d96ea56e9 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:08:21.231006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94653f-d115-4d6f-80c8-f5280719d6b6 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:21.231299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94653f-d115-4d6f-80c8-f5280719d6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:21.231599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94653f-d115-4d6f-80c8-f5280719d6b6 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:21.233200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 198/394] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:21.240992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcd76f-c5e5-4862-a762-795ea8717042 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:21.241327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcd76f-c5e5-4862-a762-795ea8717042 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:21.241611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcd76f-c5e5-4862-a762-795ea8717042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:21.274009 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c5bcd76f-c5e5-4862-a762-795ea8717042 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:21.275786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c5bcd76f-c5e5-4862-a762-795ea8717042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:21.309018 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bac5d0-4800-43ff-bc2d-c48b030cf21e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:21.390187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 199/395] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:21.391090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9b387-d9ed-452a-b331-c7f53babe72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:21.391998 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9b387-d9ed-452a-b331-c7f53babe72a None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:21.392428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9b387-d9ed-452a-b331-c7f53babe72a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:21.392863 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9b387-d9ed-452a-b331-c7f53babe72a None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88/groups/1f265fd53f4a45c387c0b90fafb42061/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:21.404511 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9b387-d9ed-452a-b331-c7f53babe72a None admin] RBAC: Authorizing `identity:create_grant(project_id=b427802e13664086974c8d53f5bfbc88, group_id=1f265fd53f4a45c387c0b90fafb42061, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:21.406570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9b387-d9ed-452a-b331-c7f53babe72a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:21.427785 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 197/396] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Oct 6 12:08:21 2022] PUT /identity/v3/projects/b427802e13664086974c8d53f5bfbc88/groups/1f265fd53f4a45c387c0b90fafb42061/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:08:21.521949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bc533-94f6-4a28-abd4-d5a92488ef32 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:21.522481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bc533-94f6-4a28-abd4-d5a92488ef32 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:21.522481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bc533-94f6-4a28-abd4-d5a92488ef32 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:21.524282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 200/397] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:21.531146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4edee1-a988-4eb8-a5b2-1c507a9b2530 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:21.531854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4edee1-a988-4eb8-a5b2-1c507a9b2530 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:21.532337 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4edee1-a988-4eb8-a5b2-1c507a9b2530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:21.553015 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ce4edee1-a988-4eb8-a5b2-1c507a9b2530 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:21.555185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ce4edee1-a988-4eb8-a5b2-1c507a9b2530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:21.576997 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:21.639165 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 198/398] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:21.716561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bc533-94f6-4a28-abd4-d5a92488ef32 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:21.734258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:21.833541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bc533-94f6-4a28-abd4-d5a92488ef32 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:21.876679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a964d0-70b8-4859-bc6a-fa41521f580e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:21.877312 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a964d0-70b8-4859-bc6a-fa41521f580e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:21.877591 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a964d0-70b8-4859-bc6a-fa41521f580e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:21.877860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a964d0-70b8-4859-bc6a-fa41521f580e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:21.891214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a964d0-70b8-4859-bc6a-fa41521f580e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:21.896239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-06a964d0-70b8-4859-bc6a-fa41521f580e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:21.896239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:21.896239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:21.896239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:21.896239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:21.896239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:21.896239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:21.896239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:21.896239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:21.896239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:21.896239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:21.896239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:21.896239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:21.896239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:21.896239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:21.896239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:21.897181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:21.897181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:21.897181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:21.897181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:21.897181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:21.897181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:21.897181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:21.897181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:21.897181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:21.897181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:21.897181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:21.897181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:21.897181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:21.897181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:21.897181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:21.897181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:21.898061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:21.898061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:21.898061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:21.898061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:21.898061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:21.898061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:21.898061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:21.898061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:21.898061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:21.898061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:21.898061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:21.898061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:21.898061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 201/399] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:21.905126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4edee1-a988-4eb8-a5b2-1c507a9b2530 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:21.936321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a964d0-70b8-4859-bc6a-fa41521f580e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:22.044142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:22.070067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4edee1-a988-4eb8-a5b2-1c507a9b2530 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:22.086983 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a964d0-70b8-4859-bc6a-fa41521f580e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:22.106266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c95b64-a9b8-4037-b500-5a8d90009e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:22.106923 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c95b64-a9b8-4037-b500-5a8d90009e6b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:22.107203 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c95b64-a9b8-4037-b500-5a8d90009e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:22.107469 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c95b64-a9b8-4037-b500-5a8d90009e6b None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:22.108171 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c95b64-a9b8-4037-b500-5a8d90009e6b None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:22.108471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c95b64-a9b8-4037-b500-5a8d90009e6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:22.109827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c95b64-a9b8-4037-b500-5a8d90009e6b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:22.119299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 199/400] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:08:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:22.124581 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c95b64-a9b8-4037-b500-5a8d90009e6b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:22.154672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:22.166529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f7190-7ddd-4827-a3b6-aba4baed5a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:22.167170 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f7190-7ddd-4827-a3b6-aba4baed5a3b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:22.167452 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f7190-7ddd-4827-a3b6-aba4baed5a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:22.167751 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f7190-7ddd-4827-a3b6-aba4baed5a3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:22.180488 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f7190-7ddd-4827-a3b6-aba4baed5a3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:22.185495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-7a3f7190-7ddd-4827-a3b6-aba4baed5a3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:08:22.185495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:22.185495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:22.185495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:22.185495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:22.185495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:22.185495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:22.185495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:22.185495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:22.185495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:22.185495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:22.185495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:22.185495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:22.185495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:22.185495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:22.185495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:22.186408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:22.186408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:22.186408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:22.186408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:22.186408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:22.186408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:22.186408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:22.186408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:22.186408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:22.186408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:22.186408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:22.186408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:22.186408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:22.186408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:22.186408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:22.186408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:22.193631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:22.193631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:22.193631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:22.193631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:22.193631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:22.193631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:22.193631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:22.193631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:22.193631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:22.193631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:22.193631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:08:22.193631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:22.193631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 202/401] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:08:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:22.193631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6cd11-5196-4f70-bf5b-7fd61189c47d None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:22.194580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6cd11-5196-4f70-bf5b-7fd61189c47d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:22.194580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6cd11-5196-4f70-bf5b-7fd61189c47d None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:22.194580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 203/402] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:22.194580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6cd11-5196-4f70-bf5b-7fd61189c47d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:22.293730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c95b64-a9b8-4037-b500-5a8d90009e6b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:22.317891 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6cd11-5196-4f70-bf5b-7fd61189c47d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:22.332359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08e3c1-7558-4da4-a5a7-647d640c40dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:22.333456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08e3c1-7558-4da4-a5a7-647d640c40dd None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:22.333916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08e3c1-7558-4da4-a5a7-647d640c40dd None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:22.334328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08e3c1-7558-4da4-a5a7-647d640c40dd None admin] PATH_INFO: `/v3/groups/1f265fd53f4a45c387c0b90fafb42061` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:22.335619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08e3c1-7558-4da4-a5a7-647d640c40dd None admin] RBAC: Authorizing `identity:get_group(group_id=1f265fd53f4a45c387c0b90fafb42061)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:22.337237 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08e3c1-7558-4da4-a5a7-647d640c40dd None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:22.343819 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 200/403] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Oct 6 12:08:22 2022] GET /identity/v3/groups/1f265fd53f4a45c387c0b90fafb42061 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:22.345266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b77e4-57a3-42d9-82f9-910f3ae63ed7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:22.345716 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b77e4-57a3-42d9-82f9-910f3ae63ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:22.346126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b77e4-57a3-42d9-82f9-910f3ae63ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:22.355953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f1be9-40ef-4c32-a0a1-c95a22f4ed60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:22.356712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f1be9-40ef-4c32-a0a1-c95a22f4ed60 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:22.357089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f1be9-40ef-4c32-a0a1-c95a22f4ed60 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:22.357460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f1be9-40ef-4c32-a0a1-c95a22f4ed60 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:22.358219 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f1be9-40ef-4c32-a0a1-c95a22f4ed60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:22.358623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f1be9-40ef-4c32-a0a1-c95a22f4ed60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:22.360130 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f1be9-40ef-4c32-a0a1-c95a22f4ed60 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:22.361625 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:22.367056 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-051b77e4-57a3-42d9-82f9-910f3ae63ed7 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:22.368380 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-051b77e4-57a3-42d9-82f9-910f3ae63ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:22.376226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 204/404] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:22.377481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f1be9-40ef-4c32-a0a1-c95a22f4ed60 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:22.447794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 201/405] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:22.448701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b77e4-57a3-42d9-82f9-910f3ae63ed7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:22.471240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:22.488718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f1be9-40ef-4c32-a0a1-c95a22f4ed60 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:22.524717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78376165-8844-44f6-891e-98cbe7737356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:22.526107 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78376165-8844-44f6-891e-98cbe7737356 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:22.526628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78376165-8844-44f6-891e-98cbe7737356 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:22.527113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78376165-8844-44f6-891e-98cbe7737356 None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:22.528592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78376165-8844-44f6-891e-98cbe7737356 None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:22.530289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78376165-8844-44f6-891e-98cbe7737356 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:22.532421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 205/406] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:22 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:22.539187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78376165-8844-44f6-891e-98cbe7737356 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:22.669286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:22.679961 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b77e4-57a3-42d9-82f9-910f3ae63ed7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:22.747449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78376165-8844-44f6-891e-98cbe7737356 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:22.757960 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494c713-341c-437e-a910-2258440194c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:22.758882 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494c713-341c-437e-a910-2258440194c4 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:22.759439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494c713-341c-437e-a910-2258440194c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:22.759973 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494c713-341c-437e-a910-2258440194c4 None admin] PATH_INFO: `/v3/groups/1f265fd53f4a45c387c0b90fafb42061` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:22.761319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494c713-341c-437e-a910-2258440194c4 None admin] RBAC: Authorizing `identity:get_group(group_id=1f265fd53f4a45c387c0b90fafb42061)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:22.762960 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494c713-341c-437e-a910-2258440194c4 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:22.764756 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 202/407] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Oct 6 12:08:22 2022] GET /identity/v3/groups/1f265fd53f4a45c387c0b90fafb42061 => generated 226 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:22.772733 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494c713-341c-437e-a910-2258440194c4 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:22.802765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c051618-7b81-4398-8581-00bea2476d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:22.803828 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c051618-7b81-4398-8581-00bea2476d40 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:22.804462 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c051618-7b81-4398-8581-00bea2476d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:22.804956 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c051618-7b81-4398-8581-00bea2476d40 None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f/groups/1f265fd53f4a45c387c0b90fafb42061/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:22.807889 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c051618-7b81-4398-8581-00bea2476d40 None admin] RBAC: Authorizing `identity:create_grant(project_id=d844147ccee64e23b49af3bf86ac4c6f, group_id=1f265fd53f4a45c387c0b90fafb42061, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:22.809558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c051618-7b81-4398-8581-00bea2476d40 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:22.828579 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 206/408] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Oct 6 12:08:22 2022] PUT /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f/groups/1f265fd53f4a45c387c0b90fafb42061/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:08:22.829958 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c051618-7b81-4398-8581-00bea2476d40 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:23.017764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:23.059614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c051618-7b81-4398-8581-00bea2476d40 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:23.065199 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494c713-341c-437e-a910-2258440194c4 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:23.107962 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:23.143615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6dd24-240d-435d-adbc-aa5b5620f57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:23.144260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6dd24-240d-435d-adbc-aa5b5620f57e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:23.144547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6dd24-240d-435d-adbc-aa5b5620f57e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:23.144826 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6dd24-240d-435d-adbc-aa5b5620f57e None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:23.149711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c54b54-93c3-42ca-b770-d84843a85e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:23.151362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c54b54-93c3-42ca-b770-d84843a85e5a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:23.151362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c54b54-93c3-42ca-b770-d84843a85e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:23.151362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c54b54-93c3-42ca-b770-d84843a85e5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:23.152749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6dd24-240d-435d-adbc-aa5b5620f57e None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:23.154601 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6dd24-240d-435d-adbc-aa5b5620f57e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:23.169503 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 207/409] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:22 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:23.170065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6dd24-240d-435d-adbc-aa5b5620f57e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:23.180299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c54b54-93c3-42ca-b770-d84843a85e5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:23.191193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-28c54b54-93c3-42ca-b770-d84843a85e5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:08:23.191193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:23.191193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:23.191193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:23.191193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:23.191193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:23.191193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:23.191193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:23.191193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:23.191193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:23.191193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:23.191193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:23.191193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:23.191193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:23.191193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:23.191193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:23.192151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:23.192151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:23.192151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:23.192151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:23.192151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:23.192151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:23.192151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:23.192151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:23.192151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:23.192151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:23.192151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:23.192151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:23.192151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:23.192151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:23.192151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:23.192151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:23.192972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:23.192972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:23.192972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:23.192972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:23.192972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:23.192972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:23.192972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:23.192972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:23.192972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:23.192972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:23.192972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:08:23.192972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:23.195814 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 203/410] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:08:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:23.204851 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c54b54-93c3-42ca-b770-d84843a85e5a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:23.387212 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6dd24-240d-435d-adbc-aa5b5620f57e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:23.478465 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a1fd2-a22e-446c-af32-54a989431945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:23.479144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a1fd2-a22e-446c-af32-54a989431945 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:23.479429 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a1fd2-a22e-446c-af32-54a989431945 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:23.479741 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a1fd2-a22e-446c-af32-54a989431945 None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f/groups/1f265fd53f4a45c387c0b90fafb42061/roles/2c11a95c1ecf4988a95aad8d96ea56e9` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:23.489474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a1fd2-a22e-446c-af32-54a989431945 None admin] RBAC: Authorizing `identity:create_grant(project_id=d844147ccee64e23b49af3bf86ac4c6f, group_id=1f265fd53f4a45c387c0b90fafb42061, role_id=2c11a95c1ecf4988a95aad8d96ea56e9)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:23.490931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a1fd2-a22e-446c-af32-54a989431945 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:23.523239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 208/411] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Oct 6 12:08:23 2022] PUT /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f/groups/1f265fd53f4a45c387c0b90fafb42061/roles/2c11a95c1ecf4988a95aad8d96ea56e9 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:08:23.541346 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c54b54-93c3-42ca-b770-d84843a85e5a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:23.591573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7ee2a-8876-44cf-a64b-c98db606a4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:23.592210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7ee2a-8876-44cf-a64b-c98db606a4fe None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:23.592492 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7ee2a-8876-44cf-a64b-c98db606a4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:23.592768 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7ee2a-8876-44cf-a64b-c98db606a4fe None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:23.593455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7ee2a-8876-44cf-a64b-c98db606a4fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:23.593762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7ee2a-8876-44cf-a64b-c98db606a4fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:23.595154 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7ee2a-8876-44cf-a64b-c98db606a4fe None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:23.608953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 204/412] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:23.614598 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a1fd2-a22e-446c-af32-54a989431945 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:23.691780 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:23.756452 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a1fd2-a22e-446c-af32-54a989431945 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:23.805512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2c7cd-5a1b-4dad-82a8-d426514cee84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:23.806270 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2c7cd-5a1b-4dad-82a8-d426514cee84 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:23.806548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2c7cd-5a1b-4dad-82a8-d426514cee84 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:23.806818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2c7cd-5a1b-4dad-82a8-d426514cee84 None admin] PATH_INFO: `/v3/groups/be0d12feeaec4b4cb3faf295204a3026` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:23.808096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b2c7cd-5a1b-4dad-82a8-d426514cee84 None admin] RBAC: Authorizing `identity:get_group(group_id=be0d12feeaec4b4cb3faf295204a3026)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:23.809615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b2c7cd-5a1b-4dad-82a8-d426514cee84 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:23.811583 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 209/413] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Oct 6 12:08:23 2022] GET /identity/v3/groups/be0d12feeaec4b4cb3faf295204a3026 => generated 229 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:23.817122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7ee2a-8876-44cf-a64b-c98db606a4fe None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:23.820183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:23.934401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ed7b0-60b2-4273-9e00-aecfc144fbb0 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:23.934776 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ed7b0-60b2-4273-9e00-aecfc144fbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:23.934950 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ed7b0-60b2-4273-9e00-aecfc144fbb0 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:23.936602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 210/414] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:08:23.942446 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf80b6-6bba-44b4-a514-b92bcd2361bb None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:23.942673 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf80b6-6bba-44b4-a514-b92bcd2361bb None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:23.942946 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf80b6-6bba-44b4-a514-b92bcd2361bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:23.948758 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7ee2a-8876-44cf-a64b-c98db606a4fe None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:23.981368 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-2caf80b6-6bba-44b4-a514-b92bcd2361bb None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:23.982569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2caf80b6-6bba-44b4-a514-b92bcd2361bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:23.995260 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b052201-0a03-4f6a-b0a6-198ce118644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:23.995919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b052201-0a03-4f6a-b0a6-198ce118644c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:23.996206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b052201-0a03-4f6a-b0a6-198ce118644c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:23.996481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b052201-0a03-4f6a-b0a6-198ce118644c None admin] PATH_INFO: `/v3/projects/5568e23a42b74b0a8d5f679d197b7521` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:23.999157 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b052201-0a03-4f6a-b0a6-198ce118644c None admin] RBAC: Authorizing `identity:get_project(project_id=5568e23a42b74b0a8d5f679d197b7521)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:24.000616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b052201-0a03-4f6a-b0a6-198ce118644c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:24.002238 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 205/415] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:23 2022] GET /identity/v3/projects/5568e23a42b74b0a8d5f679d197b7521 => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:24.007751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b052201-0a03-4f6a-b0a6-198ce118644c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:24.075233 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 211/416] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:08:24.156514 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:24.195827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b052201-0a03-4f6a-b0a6-198ce118644c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:24.229658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258703e7-571a-4047-abd8-31ee2a40bbf3 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:24.229987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258703e7-571a-4047-abd8-31ee2a40bbf3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:24.230653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258703e7-571a-4047-abd8-31ee2a40bbf3 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:24.232802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 212/417] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:08:24.237864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42884e9b-809e-4f2b-b55b-676c72b978f5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:24.238142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42884e9b-809e-4f2b-b55b-676c72b978f5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:24.238421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42884e9b-809e-4f2b-b55b-676c72b978f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:24.243158 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cee8e8-b8b2-4583-8fd4-1cb8ae81484b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:24.244371 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cee8e8-b8b2-4583-8fd4-1cb8ae81484b None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:24.244654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cee8e8-b8b2-4583-8fd4-1cb8ae81484b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:24.244926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cee8e8-b8b2-4583-8fd4-1cb8ae81484b None admin] PATH_INFO: `/v3/projects/5568e23a42b74b0a8d5f679d197b7521/groups/be0d12feeaec4b4cb3faf295204a3026/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:24.251315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cee8e8-b8b2-4583-8fd4-1cb8ae81484b None admin] RBAC: Authorizing `identity:create_grant(project_id=5568e23a42b74b0a8d5f679d197b7521, group_id=be0d12feeaec4b4cb3faf295204a3026, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:24.252825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cee8e8-b8b2-4583-8fd4-1cb8ae81484b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:24.269183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-42884e9b-809e-4f2b-b55b-676c72b978f5 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:24.271046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-42884e9b-809e-4f2b-b55b-676c72b978f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:24.276085 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 206/418] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Oct 6 12:08:24 2022] PUT /identity/v3/projects/5568e23a42b74b0a8d5f679d197b7521/groups/be0d12feeaec4b4cb3faf295204a3026/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:08:24.327108 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:24.346555 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 213/419] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:24.507852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cee8e8-b8b2-4583-8fd4-1cb8ae81484b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:24.628925 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cee8e8-b8b2-4583-8fd4-1cb8ae81484b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:24.660630 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:24.668198 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d06553-9409-4565-a580-1a7a9453b776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:24.669217 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d06553-9409-4565-a580-1a7a9453b776 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:24.669529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d06553-9409-4565-a580-1a7a9453b776 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:24.669803 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d06553-9409-4565-a580-1a7a9453b776 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:24.686009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d06553-9409-4565-a580-1a7a9453b776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:24.691969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-94d06553-9409-4565-a580-1a7a9453b776 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:08:24.691969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:24.691969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:24.691969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:24.691969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:24.691969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:24.691969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:24.691969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:24.691969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:24.691969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:24.691969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:24.691969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:24.691969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:24.691969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:24.691969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:24.691969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:24.692868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:24.692868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:24.692868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:24.692868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:24.692868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:24.692868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:24.692868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:24.692868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:24.692868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:24.692868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:24.692868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:24.692868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:24.692868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:24.692868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:24.692868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:24.692868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:24.694084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:24.694084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:24.694084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:24.694084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:24.694084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:24.694084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:24.694084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:24.694084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:24.694084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:24.694084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:24.694084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:08:24.694084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:24.694084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 207/420] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:08:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:24.699858 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42884e9b-809e-4f2b-b55b-676c72b978f5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:24.761128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d06553-9409-4565-a580-1a7a9453b776 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:24.850331 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42884e9b-809e-4f2b-b55b-676c72b978f5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:24.890490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa66bc5-9fb7-4554-8ad4-a8b33b502f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:24.891101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa66bc5-9fb7-4554-8ad4-a8b33b502f32 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:24.891378 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa66bc5-9fb7-4554-8ad4-a8b33b502f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:24.891673 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa66bc5-9fb7-4554-8ad4-a8b33b502f32 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:24.892336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa66bc5-9fb7-4554-8ad4-a8b33b502f32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:24.892637 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa66bc5-9fb7-4554-8ad4-a8b33b502f32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:24.894006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa66bc5-9fb7-4554-8ad4-a8b33b502f32 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:24.913018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 214/421] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:24.920343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa66bc5-9fb7-4554-8ad4-a8b33b502f32 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:24.928503 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d06553-9409-4565-a580-1a7a9453b776 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:25.000529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fb80c-0192-402a-be9a-f8b662fb18f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:25.001164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fb80c-0192-402a-be9a-f8b662fb18f9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:25.001440 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fb80c-0192-402a-be9a-f8b662fb18f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:25.001706 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fb80c-0192-402a-be9a-f8b662fb18f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:25.009883 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:25.026843 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fb80c-0192-402a-be9a-f8b662fb18f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:25.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-b80fb80c-0192-402a-be9a-f8b662fb18f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:25.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:25.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:25.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:25.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:25.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:25.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:25.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:25.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:25.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:25.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:25.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:25.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:25.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:25.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:25.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:25.033389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:25.033389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:25.033389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:25.033389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:25.033389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:25.033389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:25.033389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:25.033389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:25.033389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:25.033389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:25.033389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:25.033389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:25.033389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:25.033389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:25.033389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:25.033389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:25.034213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:25.034213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:25.034213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:25.034213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:25.034213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:25.034213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:25.034213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:25.034213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:25.034213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:25.034213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:25.034213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:25.034213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:25.043775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 208/422] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:25.053147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fb80c-0192-402a-be9a-f8b662fb18f9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:25.121388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa66bc5-9fb7-4554-8ad4-a8b33b502f32 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:25.135441 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:25.184220 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626ed65-1b3f-4680-ab73-ab0d682beaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:25.185201 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626ed65-1b3f-4680-ab73-ab0d682beaf3 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:25.185680 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626ed65-1b3f-4680-ab73-ab0d682beaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:25.186177 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626ed65-1b3f-4680-ab73-ab0d682beaf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:25.256228 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fb80c-0192-402a-be9a-f8b662fb18f9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:25.293514 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626ed65-1b3f-4680-ab73-ab0d682beaf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:25.295286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626ed65-1b3f-4680-ab73-ab0d682beaf3 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:25.329461 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5797162-0d97-4e6e-8ee8-f9bf5a84cbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:25.330076 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5797162-0d97-4e6e-8ee8-f9bf5a84cbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:25.330357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5797162-0d97-4e6e-8ee8-f9bf5a84cbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:25.330628 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5797162-0d97-4e6e-8ee8-f9bf5a84cbe9 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:25.331295 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5797162-0d97-4e6e-8ee8-f9bf5a84cbe9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:25.331624 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5797162-0d97-4e6e-8ee8-f9bf5a84cbe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:25.333014 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5797162-0d97-4e6e-8ee8-f9bf5a84cbe9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:25.341811 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 209/423] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:08:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:25.342538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:25.344099 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814b222-63fa-41a8-8bc7-4def9cdca09b None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:25.344568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814b222-63fa-41a8-8bc7-4def9cdca09b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:25.345078 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814b222-63fa-41a8-8bc7-4def9cdca09b None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:25.350773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 210/424] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:25.351797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814b222-63fa-41a8-8bc7-4def9cdca09b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:25.404382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 215/425] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:25.406227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c1da1-027b-4399-989d-bb0a193bd060 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:25.406581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c1da1-027b-4399-989d-bb0a193bd060 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:25.406921 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c1da1-027b-4399-989d-bb0a193bd060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:25.426506 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-a02c1da1-027b-4399-989d-bb0a193bd060 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:25.427845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a02c1da1-027b-4399-989d-bb0a193bd060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:25.473477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814b222-63fa-41a8-8bc7-4def9cdca09b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:25.492591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:25.511538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 216/426] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:25.511538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c1da1-027b-4399-989d-bb0a193bd060 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:25.539296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1459c8-e19e-45b9-9b1f-28a311a0fb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:25.541357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1459c8-e19e-45b9-9b1f-28a311a0fb9d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:25.541702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1459c8-e19e-45b9-9b1f-28a311a0fb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:25.542086 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1459c8-e19e-45b9-9b1f-28a311a0fb9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:25.655949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:25.669537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1459c8-e19e-45b9-9b1f-28a311a0fb9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:25.671909 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1459c8-e19e-45b9-9b1f-28a311a0fb9d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:25.692509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c1da1-027b-4399-989d-bb0a193bd060 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:25.739938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d523044-f699-4e4c-a90f-0c72183a29f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:25.740558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d523044-f699-4e4c-a90f-0c72183a29f5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:25.740846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d523044-f699-4e4c-a90f-0c72183a29f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:25.741119 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d523044-f699-4e4c-a90f-0c72183a29f5 None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:25.742500 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d523044-f699-4e4c-a90f-0c72183a29f5 None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:25.744895 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d523044-f699-4e4c-a90f-0c72183a29f5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:25.748838 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 217/427] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:25 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:25.755663 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d523044-f699-4e4c-a90f-0c72183a29f5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:25.868286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 211/428] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:25.869298 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1459c8-e19e-45b9-9b1f-28a311a0fb9d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:25.886096 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:25.919548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d523044-f699-4e4c-a90f-0c72183a29f5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:25.960150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4db58-d55e-48d8-a947-093f38ad0cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:25.960783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4db58-d55e-48d8-a947-093f38ad0cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:25.961071 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4db58-d55e-48d8-a947-093f38ad0cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:25.961331 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4db58-d55e-48d8-a947-093f38ad0cf1 None admin] PATH_INFO: `/v3/groups/1f265fd53f4a45c387c0b90fafb42061` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:25.962971 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4db58-d55e-48d8-a947-093f38ad0cf1 None admin] RBAC: Authorizing `identity:get_group(group_id=1f265fd53f4a45c387c0b90fafb42061)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:25.964465 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4db58-d55e-48d8-a947-093f38ad0cf1 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:25.966402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 218/429] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Oct 6 12:08:25 2022] GET /identity/v3/groups/1f265fd53f4a45c387c0b90fafb42061 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:25.967324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4db58-d55e-48d8-a947-093f38ad0cf1 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:25.983151 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:26.001392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1459c8-e19e-45b9-9b1f-28a311a0fb9d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:26.037720 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5116b3-8178-4681-9934-271b7de5440d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:26.038341 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5116b3-8178-4681-9934-271b7de5440d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:26.038614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5116b3-8178-4681-9934-271b7de5440d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:26.038885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5116b3-8178-4681-9934-271b7de5440d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:26.049589 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5116b3-8178-4681-9934-271b7de5440d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:26.054271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-6c5116b3-8178-4681-9934-271b7de5440d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:26.054271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:26.054271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:26.054271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:26.054271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:26.054271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:26.054271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:26.054271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:26.054271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:26.054271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:26.054271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:26.054271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:26.054271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:26.054271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:26.054271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:26.054271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:26.055156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:26.055156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:26.055156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:26.055156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:26.055156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:26.055156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:26.055156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:26.055156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:26.055156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:26.055156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:26.055156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:26.055156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:26.055156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:26.055156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:26.055156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:26.055156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:26.060583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:26.060583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:26.060583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:26.060583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:26.060583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:26.060583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:26.060583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:26.060583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:26.060583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:26.060583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:26.060583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 12:08:26.060583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:26.060583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 212/430] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:26.060583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5116b3-8178-4681-9934-271b7de5440d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:26.088089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4db58-d55e-48d8-a947-093f38ad0cf1 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:26.124393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670e760-ba81-455a-a1bb-2f7e6a14a8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:26.125024 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670e760-ba81-455a-a1bb-2f7e6a14a8fa None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:26.125305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670e760-ba81-455a-a1bb-2f7e6a14a8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:26.125580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670e760-ba81-455a-a1bb-2f7e6a14a8fa None admin] PATH_INFO: `/v3/projects/b427802e13664086974c8d53f5bfbc88` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:26.126709 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670e760-ba81-455a-a1bb-2f7e6a14a8fa None admin] RBAC: Authorizing `identity:get_project(project_id=b427802e13664086974c8d53f5bfbc88)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:26.128250 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670e760-ba81-455a-a1bb-2f7e6a14a8fa None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:26.131548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 219/431] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:25 2022] GET /identity/v3/projects/b427802e13664086974c8d53f5bfbc88 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:26.131548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9726697-b722-4223-8f47-61dde0819b4f None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:26.131548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9726697-b722-4223-8f47-61dde0819b4f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:26.131548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9726697-b722-4223-8f47-61dde0819b4f None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:26.137495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 220/432] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:26.137495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9726697-b722-4223-8f47-61dde0819b4f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:26.146294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:26.182308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5116b3-8178-4681-9934-271b7de5440d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:26.217082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:26.218537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b95fa-83cc-4709-8b85-67de6c661e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:26.219271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b95fa-83cc-4709-8b85-67de6c661e46 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:26.219452 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b95fa-83cc-4709-8b85-67de6c661e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:26.219779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b95fa-83cc-4709-8b85-67de6c661e46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:26.220554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b95fa-83cc-4709-8b85-67de6c661e46 None admin] RBAC: Adding query filter params (group.id=1f265fd53f4a45c387c0b90fafb42061, role.id=2c11a95c1ecf4988a95aad8d96ea56e9, scope.project.id=b427802e13664086974c8d53f5bfbc88) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:26.220950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b95fa-83cc-4709-8b85-67de6c661e46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:26.222745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b95fa-83cc-4709-8b85-67de6c661e46 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:26.238374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 213/433] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Oct 6 12:08:26 2022] GET /identity/v3/role_assignments?group.id=1f265fd53f4a45c387c0b90fafb42061&scope.project.id=b427802e13664086974c8d53f5bfbc88&role.id=2c11a95c1ecf4988a95aad8d96ea56e9 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:26.238374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b95fa-83cc-4709-8b85-67de6c661e46 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:26.259187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9726697-b722-4223-8f47-61dde0819b4f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:26.305357 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadc1e9-c713-4838-bdbe-5c8ec669dcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:26.307169 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadc1e9-c713-4838-bdbe-5c8ec669dcce None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:26.307448 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadc1e9-c713-4838-bdbe-5c8ec669dcce None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:26.307744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadc1e9-c713-4838-bdbe-5c8ec669dcce None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:26.308710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadc1e9-c713-4838-bdbe-5c8ec669dcce None admin] RBAC: Adding query filter params (name=member) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:26.309011 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadc1e9-c713-4838-bdbe-5c8ec669dcce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:26.310647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadc1e9-c713-4838-bdbe-5c8ec669dcce None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:26.322441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 221/434] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:08:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:26.323867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c22ca-d1c7-4245-aa8b-c6cad9057af9 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:26.324153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c22ca-d1c7-4245-aa8b-c6cad9057af9 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:26.324425 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c22ca-d1c7-4245-aa8b-c6cad9057af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:26.345490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-2b1c22ca-d1c7-4245-aa8b-c6cad9057af9 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:26.346829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2b1c22ca-d1c7-4245-aa8b-c6cad9057af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:26.364860 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b95fa-83cc-4709-8b85-67de6c661e46 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:26.407557 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f3139-2323-43b8-8502-6bc6120dbd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:26.408330 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f3139-2323-43b8-8502-6bc6120dbd15 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:26.408930 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f3139-2323-43b8-8502-6bc6120dbd15 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:26.409271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f3139-2323-43b8-8502-6bc6120dbd15 None admin] PATH_INFO: `/v3/groups/1f265fd53f4a45c387c0b90fafb42061` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:26.410869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f3139-2323-43b8-8502-6bc6120dbd15 None admin] RBAC: Authorizing `identity:get_group(group_id=1f265fd53f4a45c387c0b90fafb42061)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:26.412971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f3139-2323-43b8-8502-6bc6120dbd15 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:26.415731 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 214/435] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Oct 6 12:08:26 2022] GET /identity/v3/groups/1f265fd53f4a45c387c0b90fafb42061 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:26.417050 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f3139-2323-43b8-8502-6bc6120dbd15 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:26.421883 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:26.435909 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 222/436] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:26.438334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c22ca-d1c7-4245-aa8b-c6cad9057af9 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:26.529061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:26.557794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f3139-2323-43b8-8502-6bc6120dbd15 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:26.567567 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c22ca-d1c7-4245-aa8b-c6cad9057af9 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:26.608422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed426c-0874-4e30-923c-6329f20e4b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:26.609452 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed426c-0874-4e30-923c-6329f20e4b1d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:26.609980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed426c-0874-4e30-923c-6329f20e4b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:26.610450 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed426c-0874-4e30-923c-6329f20e4b1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:26.619156 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf225aee-bd8c-4e33-9643-4996d9ccc9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:26.619993 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf225aee-bd8c-4e33-9643-4996d9ccc9bd None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:26.620102 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf225aee-bd8c-4e33-9643-4996d9ccc9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:26.620339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf225aee-bd8c-4e33-9643-4996d9ccc9bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:26.621075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf225aee-bd8c-4e33-9643-4996d9ccc9bd None admin] RBAC: Adding query filter params (group.id=1f265fd53f4a45c387c0b90fafb42061, role.id=bbea734b8a704d7ca8ec61bd45536cbe, scope.project.id=b427802e13664086974c8d53f5bfbc88) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:26.621388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf225aee-bd8c-4e33-9643-4996d9ccc9bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:26.623626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf225aee-bd8c-4e33-9643-4996d9ccc9bd None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:26.632051 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 223/437] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Oct 6 12:08:26 2022] GET /identity/v3/role_assignments?group.id=1f265fd53f4a45c387c0b90fafb42061&scope.project.id=b427802e13664086974c8d53f5bfbc88&role.id=bbea734b8a704d7ca8ec61bd45536cbe => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:26.698379 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed426c-0874-4e30-923c-6329f20e4b1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:26.702425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed426c-0874-4e30-923c-6329f20e4b1d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:26.706078 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:26.726188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf225aee-bd8c-4e33-9643-4996d9ccc9bd None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:26.845914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 215/438] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:26.851805 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed426c-0874-4e30-923c-6329f20e4b1d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:26.861207 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf225aee-bd8c-4e33-9643-4996d9ccc9bd None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:26.894898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:26.901602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700b460-4fc5-4afc-adaf-f2278ff26b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:26.902459 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700b460-4fc5-4afc-adaf-f2278ff26b23 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:26.902646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700b460-4fc5-4afc-adaf-f2278ff26b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:26.902923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700b460-4fc5-4afc-adaf-f2278ff26b23 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:26.917296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700b460-4fc5-4afc-adaf-f2278ff26b23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:26.922685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-a700b460-4fc5-4afc-adaf-f2278ff26b23 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:08:26.922685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:26.922685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:26.922685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:26.922685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:26.922685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:26.922685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:26.922685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:26.922685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:26.922685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:26.922685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:26.922685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:26.922685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:26.922685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:26.922685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:26.922685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:26.923757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:26.923757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:26.923757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:26.923757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:26.923757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:26.923757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:26.923757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:26.923757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:26.923757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:26.923757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:26.923757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:26.923757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:26.923757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:26.923757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:26.923757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:26.923757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:26.924594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:26.924594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:26.924594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:26.924594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:26.924594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:26.924594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:26.924594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:26.924594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:26.924594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:26.924594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:26.924594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:08:26.924594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:26.925237 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 224/439] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:08:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:26.926323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072cd9c-7ca8-4f61-b456-21d797b57c1d None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:26.926611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072cd9c-7ca8-4f61-b456-21d797b57c1d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:26.926884 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072cd9c-7ca8-4f61-b456-21d797b57c1d None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:26.931652 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 225/440] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:26.931652 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072cd9c-7ca8-4f61-b456-21d797b57c1d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:26.964595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed426c-0874-4e30-923c-6329f20e4b1d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:26.968103 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:27.013320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba13f9-bed3-4ec0-8f51-bba5b62fae0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:27.013928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba13f9-bed3-4ec0-8f51-bba5b62fae0a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:27.013928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba13f9-bed3-4ec0-8f51-bba5b62fae0a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:27.014090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba13f9-bed3-4ec0-8f51-bba5b62fae0a None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:27.015257 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba13f9-bed3-4ec0-8f51-bba5b62fae0a None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:27.016790 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba13f9-bed3-4ec0-8f51-bba5b62fae0a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:27.018730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 216/441] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:26 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:27.019906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf81150-e284-427d-b270-2bdc891bf8e1 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:27.020188 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf81150-e284-427d-b270-2bdc891bf8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:27.020460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf81150-e284-427d-b270-2bdc891bf8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:27.042055 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-acf81150-e284-427d-b270-2bdc891bf8e1 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:27.043243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-acf81150-e284-427d-b270-2bdc891bf8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:27.060211 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072cd9c-7ca8-4f61-b456-21d797b57c1d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:27.102042 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f2d6c-6f6c-4647-858b-bb88be1831af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:27.103956 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f2d6c-6f6c-4647-858b-bb88be1831af None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:27.104238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f2d6c-6f6c-4647-858b-bb88be1831af None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:27.104508 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f2d6c-6f6c-4647-858b-bb88be1831af None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:27.105236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f2d6c-6f6c-4647-858b-bb88be1831af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:27.105543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f2d6c-6f6c-4647-858b-bb88be1831af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:27.107086 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f2d6c-6f6c-4647-858b-bb88be1831af None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:27.115834 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 226/442] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:27.118654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 217/443] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:27.120557 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f2d6c-6f6c-4647-858b-bb88be1831af None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:27.125134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf81150-e284-427d-b270-2bdc891bf8e1 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:27.185286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:27.253767 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f2d6c-6f6c-4647-858b-bb88be1831af None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:27.260330 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf81150-e284-427d-b270-2bdc891bf8e1 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:27.264612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:27.301590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93fc50-4ae9-44f8-aa68-a3cc628e0ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:27.302526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93fc50-4ae9-44f8-aa68-a3cc628e0ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:27.303051 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93fc50-4ae9-44f8-aa68-a3cc628e0ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:27.303584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93fc50-4ae9-44f8-aa68-a3cc628e0ac5 None admin] PATH_INFO: `/v3/groups/1f265fd53f4a45c387c0b90fafb42061` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:27.304984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93fc50-4ae9-44f8-aa68-a3cc628e0ac5 None admin] RBAC: Authorizing `identity:get_group(group_id=1f265fd53f4a45c387c0b90fafb42061)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:27.306382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bb280-f65d-4e8a-85dc-30764e285eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:27.307057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bb280-f65d-4e8a-85dc-30764e285eca None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:27.307403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bb280-f65d-4e8a-85dc-30764e285eca None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:27.307815 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bb280-f65d-4e8a-85dc-30764e285eca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:27.308055 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93fc50-4ae9-44f8-aa68-a3cc628e0ac5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:27.309839 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 227/444] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Oct 6 12:08:27 2022] GET /identity/v3/groups/1f265fd53f4a45c387c0b90fafb42061 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:27.315613 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93fc50-4ae9-44f8-aa68-a3cc628e0ac5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:27.385478 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bb280-f65d-4e8a-85dc-30764e285eca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:27.386947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bb280-f65d-4e8a-85dc-30764e285eca None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:27.424944 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93fc50-4ae9-44f8-aa68-a3cc628e0ac5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:27.446331 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:27.458888 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb78c1-fc96-41a7-a557-4f09d64ae4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:27.459493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb78c1-fc96-41a7-a557-4f09d64ae4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:27.459805 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb78c1-fc96-41a7-a557-4f09d64ae4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:27.460080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb78c1-fc96-41a7-a557-4f09d64ae4f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:27.460801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb78c1-fc96-41a7-a557-4f09d64ae4f0 None admin] RBAC: Adding query filter params (group.id=1f265fd53f4a45c387c0b90fafb42061, role.id=bbea734b8a704d7ca8ec61bd45536cbe, scope.project.id=d844147ccee64e23b49af3bf86ac4c6f) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:27.461109 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb78c1-fc96-41a7-a557-4f09d64ae4f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:27.463019 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb78c1-fc96-41a7-a557-4f09d64ae4f0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:27.468772 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 228/445] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Oct 6 12:08:27 2022] GET /identity/v3/role_assignments?group.id=1f265fd53f4a45c387c0b90fafb42061&scope.project.id=d844147ccee64e23b49af3bf86ac4c6f&role.id=bbea734b8a704d7ca8ec61bd45536cbe => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:27.469382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb78c1-fc96-41a7-a557-4f09d64ae4f0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:27.504044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 218/446] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 12:08:27.509842 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bb280-f65d-4e8a-85dc-30764e285eca None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:27.512074 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:27.586505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb78c1-fc96-41a7-a557-4f09d64ae4f0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:27.625936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c0249-57ee-40d1-b8fa-c6e23b338628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:27.626587 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c0249-57ee-40d1-b8fa-c6e23b338628 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:27.626870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c0249-57ee-40d1-b8fa-c6e23b338628 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:27.627145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c0249-57ee-40d1-b8fa-c6e23b338628 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:27.636626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bb280-f65d-4e8a-85dc-30764e285eca None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:27.640083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c0249-57ee-40d1-b8fa-c6e23b338628 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:27.645546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-d24c0249-57ee-40d1-b8fa-c6e23b338628 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:08:27.645546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:27.645546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:27.645546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:27.645546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:27.645546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:27.645546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:27.645546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:27.645546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:27.645546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:27.645546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:27.645546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:27.645546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:27.645546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:27.645546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:27.645546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:27.646762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:27.646762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:27.646762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:27.646762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:27.646762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:27.646762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:27.646762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:27.646762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:27.646762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:27.646762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:27.646762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:27.646762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:27.646762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:27.646762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:27.646762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:27.646762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:27.647903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:27.647903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:27.647903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:27.647903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:27.647903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:27.647903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:27.647903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:27.647903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:27.647903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:27.647903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:27.647903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:08:27.647903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:27.647903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 229/447] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:08:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:27.653117 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c0249-57ee-40d1-b8fa-c6e23b338628 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:27.680740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95f4f7-911c-4095-afc8-0c1102068bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:27.681367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95f4f7-911c-4095-afc8-0c1102068bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:27.681644 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95f4f7-911c-4095-afc8-0c1102068bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:27.681913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95f4f7-911c-4095-afc8-0c1102068bf8 None admin] PATH_INFO: `/v3/projects/d844147ccee64e23b49af3bf86ac4c6f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:27.683028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95f4f7-911c-4095-afc8-0c1102068bf8 None admin] RBAC: Authorizing `identity:get_project(project_id=d844147ccee64e23b49af3bf86ac4c6f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:27.684629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95f4f7-911c-4095-afc8-0c1102068bf8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:27.686248 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 219/448] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:27 2022] GET /identity/v3/projects/d844147ccee64e23b49af3bf86ac4c6f => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:27.692066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95f4f7-911c-4095-afc8-0c1102068bf8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:27.711394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:27.770903 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:27.777959 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c0249-57ee-40d1-b8fa-c6e23b338628 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:27.809390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95f4f7-911c-4095-afc8-0c1102068bf8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:27.819604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b333bf1-989e-4de6-822b-f552315889f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:27.820201 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b333bf1-989e-4de6-822b-f552315889f6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:27.820478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b333bf1-989e-4de6-822b-f552315889f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:27.820746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b333bf1-989e-4de6-822b-f552315889f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:27.821928 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b333bf1-989e-4de6-822b-f552315889f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:27.822403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b333bf1-989e-4de6-822b-f552315889f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:27.823678 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b333bf1-989e-4de6-822b-f552315889f6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:27.833883 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 230/449] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:27.840083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b333bf1-989e-4de6-822b-f552315889f6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:27.849573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53b153-6335-47e5-b204-1f0bd6a91e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:27.850441 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b153-6335-47e5-b204-1f0bd6a91e65 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:27.850856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b153-6335-47e5-b204-1f0bd6a91e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:27.851310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b153-6335-47e5-b204-1f0bd6a91e65 None admin] PATH_INFO: `/v3/groups/1f265fd53f4a45c387c0b90fafb42061` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:27.854051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53b153-6335-47e5-b204-1f0bd6a91e65 None admin] RBAC: Authorizing `identity:get_group(group_id=1f265fd53f4a45c387c0b90fafb42061)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:27.855493 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53b153-6335-47e5-b204-1f0bd6a91e65 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:27.857148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 220/450] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Oct 6 12:08:27 2022] GET /identity/v3/groups/1f265fd53f4a45c387c0b90fafb42061 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:27.863330 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53b153-6335-47e5-b204-1f0bd6a91e65 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:27.946447 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b333bf1-989e-4de6-822b-f552315889f6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:27.955068 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:27.960567 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53b153-6335-47e5-b204-1f0bd6a91e65 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:27.980646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db98a7-542f-446c-b711-c5127f96b708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:27.981254 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db98a7-542f-446c-b711-c5127f96b708 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:27.981537 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db98a7-542f-446c-b711-c5127f96b708 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:27.981809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db98a7-542f-446c-b711-c5127f96b708 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:27.994016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b9923-0149-46b4-8536-bfc826f2057d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:27.994386 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b9923-0149-46b4-8536-bfc826f2057d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:27.994675 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b9923-0149-46b4-8536-bfc826f2057d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:27.994943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b9923-0149-46b4-8536-bfc826f2057d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:27.995686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b9923-0149-46b4-8536-bfc826f2057d None admin] RBAC: Adding query filter params (group.id=1f265fd53f4a45c387c0b90fafb42061, role.id=2c11a95c1ecf4988a95aad8d96ea56e9, scope.project.id=d844147ccee64e23b49af3bf86ac4c6f) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:27.995993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b9923-0149-46b4-8536-bfc826f2057d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:27.997426 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b9923-0149-46b4-8536-bfc826f2057d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:28.003015 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 221/451] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Oct 6 12:08:27 2022] GET /identity/v3/role_assignments?group.id=1f265fd53f4a45c387c0b90fafb42061&scope.project.id=d844147ccee64e23b49af3bf86ac4c6f&role.id=2c11a95c1ecf4988a95aad8d96ea56e9 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:28.052004 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db98a7-542f-446c-b711-c5127f96b708 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:28.053420 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db98a7-542f-446c-b711-c5127f96b708 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:28.157539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 231/452] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:28.163344 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b9923-0149-46b4-8536-bfc826f2057d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:28.166337 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:28.268693 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b9923-0149-46b4-8536-bfc826f2057d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:28.303614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb351c-0258-4fbf-b937-53177b552590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:28.304241 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb351c-0258-4fbf-b937-53177b552590 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:28.304528 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb351c-0258-4fbf-b937-53177b552590 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:28.304806 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb351c-0258-4fbf-b937-53177b552590 None admin] PATH_INFO: `/v3/projects/5568e23a42b74b0a8d5f679d197b7521` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:28.306450 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb351c-0258-4fbf-b937-53177b552590 None admin] RBAC: Authorizing `identity:get_project(project_id=5568e23a42b74b0a8d5f679d197b7521)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:28.307892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb351c-0258-4fbf-b937-53177b552590 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:28.309613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 222/453] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:08:28 2022] GET /identity/v3/projects/5568e23a42b74b0a8d5f679d197b7521 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:28.315285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db98a7-542f-446c-b711-c5127f96b708 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:28.325764 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:28.412626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db98a7-542f-446c-b711-c5127f96b708 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:28.445115 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72354cf-ac9c-4c17-8002-52bf4ad5caae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:28.445767 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72354cf-ac9c-4c17-8002-52bf4ad5caae None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:28.446049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72354cf-ac9c-4c17-8002-52bf4ad5caae None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:28.446349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72354cf-ac9c-4c17-8002-52bf4ad5caae None admin] PATH_INFO: `/v3/groups/be0d12feeaec4b4cb3faf295204a3026` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:28.447458 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72354cf-ac9c-4c17-8002-52bf4ad5caae None admin] RBAC: Authorizing `identity:get_group(group_id=be0d12feeaec4b4cb3faf295204a3026)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:28.448930 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72354cf-ac9c-4c17-8002-52bf4ad5caae None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:28.450507 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 232/454] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Oct 6 12:08:28 2022] GET /identity/v3/groups/be0d12feeaec4b4cb3faf295204a3026 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:28.456315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb351c-0258-4fbf-b937-53177b552590 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:28.521586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:28.551124 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb351c-0258-4fbf-b937-53177b552590 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:28.584132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24a523-ee70-4881-b3d2-416298f26eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:28.584660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24a523-ee70-4881-b3d2-416298f26eca None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:28.584936 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24a523-ee70-4881-b3d2-416298f26eca None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:28.585203 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24a523-ee70-4881-b3d2-416298f26eca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:28.585907 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24a523-ee70-4881-b3d2-416298f26eca None admin] RBAC: Adding query filter params (group.id=be0d12feeaec4b4cb3faf295204a3026, role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=5568e23a42b74b0a8d5f679d197b7521) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:28.586209 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24a523-ee70-4881-b3d2-416298f26eca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:28.587693 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24a523-ee70-4881-b3d2-416298f26eca None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:28.593463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 223/455] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Oct 6 12:08:28 2022] GET /identity/v3/role_assignments?group.id=be0d12feeaec4b4cb3faf295204a3026&scope.project.id=5568e23a42b74b0a8d5f679d197b7521&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:31.253549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c34154-f8e5-4e68-b1d3-b9ff797973c8 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:31.254010 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c34154-f8e5-4e68-b1d3-b9ff797973c8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:31.254116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c34154-f8e5-4e68-b1d3-b9ff797973c8 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:31.255742 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 233/456] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:31.260952 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d0043-821a-4f9f-be78-245e6f9c3195 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:31.261353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d0043-821a-4f9f-be78-245e6f9c3195 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:31.261524 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d0043-821a-4f9f-be78-245e6f9c3195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:31.285015 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-881d0043-821a-4f9f-be78-245e6f9c3195 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:31.286210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-881d0043-821a-4f9f-be78-245e6f9c3195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:31.384863 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 224/457] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:31.490599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a0b67-1b46-4532-9981-ea87ee0bcf24 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:31.491242 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a0b67-1b46-4532-9981-ea87ee0bcf24 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:31.491779 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a0b67-1b46-4532-9981-ea87ee0bcf24 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:31.494217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 234/458] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:31.497805 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e91d8-1486-4f4d-b048-22095424b586 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:31.498090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e91d8-1486-4f4d-b048-22095424b586 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:31.498363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e91d8-1486-4f4d-b048-22095424b586 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:31.499704 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 225/459] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:31.500741 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce2f26-48c3-452b-a519-de6bd8acb9ee None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:31.501228 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce2f26-48c3-452b-a519-de6bd8acb9ee None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:31.501698 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce2f26-48c3-452b-a519-de6bd8acb9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:31.504500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221a732-296b-4bc3-982a-263d124b3a0d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:31.504995 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221a732-296b-4bc3-982a-263d124b3a0d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:31.505421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221a732-296b-4bc3-982a-263d124b3a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:31.535531 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f0ce2f26-48c3-452b-a519-de6bd8acb9ee None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:31.535531 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f0ce2f26-48c3-452b-a519-de6bd8acb9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:31.535777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f221a732-296b-4bc3-982a-263d124b3a0d None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:31.536789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f221a732-296b-4bc3-982a-263d124b3a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:31.623977 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:31.628769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 226/460] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:08:31.713695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 235/461] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:31.720378 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221a732-296b-4bc3-982a-263d124b3a0d None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:31.858314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221a732-296b-4bc3-982a-263d124b3a0d None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:31.908564 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b74f26-176f-4a88-b388-83629e0a41c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:31.909202 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b74f26-176f-4a88-b388-83629e0a41c7 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:31.909483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b74f26-176f-4a88-b388-83629e0a41c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:31.909754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b74f26-176f-4a88-b388-83629e0a41c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:31.920016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b74f26-176f-4a88-b388-83629e0a41c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:31.921481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b74f26-176f-4a88-b388-83629e0a41c7 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:31.938497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-37b74f26-176f-4a88-b388-83629e0a41c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:31.938497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:31.938497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:31.938497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:31.938497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:31.938497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:31.938497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:31.938497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:31.938497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:31.938497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:31.938497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:31.938497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:31.938497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:31.938497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:31.938497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:31.938497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:31.939445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:31.940586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:31.940586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:31.940586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:31.940586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 227/462] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:31.945659 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce2f26-48c3-452b-a519-de6bd8acb9ee None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:31.954985 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:31.994792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b98b4-4543-49e7-9eaf-b77a249ae15b None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:31.995417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b98b4-4543-49e7-9eaf-b77a249ae15b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:31.995945 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b98b4-4543-49e7-9eaf-b77a249ae15b None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:31.997493 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 228/463] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:32.006895 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b98b4-4543-49e7-9eaf-b77a249ae15b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:32.063679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:32.065236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce2f26-48c3-452b-a519-de6bd8acb9ee None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:32.103142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8fb4bb-026a-4ee1-a00e-54579ddbb8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:32.103773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fb4bb-026a-4ee1-a00e-54579ddbb8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:32.104047 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fb4bb-026a-4ee1-a00e-54579ddbb8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:32.104311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fb4bb-026a-4ee1-a00e-54579ddbb8c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:32.104943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8fb4bb-026a-4ee1-a00e-54579ddbb8c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:32.105240 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8fb4bb-026a-4ee1-a00e-54579ddbb8c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:32.106603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8fb4bb-026a-4ee1-a00e-54579ddbb8c2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:32.123012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 236/464] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:32.123012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c20e7-98b0-4247-8e57-3cd7ac17ad56 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:32.123012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c20e7-98b0-4247-8e57-3cd7ac17ad56 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:32.123012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c20e7-98b0-4247-8e57-3cd7ac17ad56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:32.132287 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b98b4-4543-49e7-9eaf-b77a249ae15b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:32.138159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3f4c20e7-98b0-4247-8e57-3cd7ac17ad56 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:32.139340 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3f4c20e7-98b0-4247-8e57-3cd7ac17ad56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:32.167629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22eeac-15b7-4c51-acbd-a3378739a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:32.168258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22eeac-15b7-4c51-acbd-a3378739a5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:32.168552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22eeac-15b7-4c51-acbd-a3378739a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:32.168804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22eeac-15b7-4c51-acbd-a3378739a5e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:32.178994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f22eeac-15b7-4c51-acbd-a3378739a5e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:32.180588 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f22eeac-15b7-4c51-acbd-a3378739a5e9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:32.195808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-0f22eeac-15b7-4c51-acbd-a3378739a5e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:32.195808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:32.195808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:32.195808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:32.195808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:32.195808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:32.195808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:32.195808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:32.195808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:32.195808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:32.195808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:32.195808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:32.195808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:32.195808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:32.195808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:32.195808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:32.197302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:32.198166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:32.198166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:32.198166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:32.198166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 229/465] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:08:32.198831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22eeac-15b7-4c51-acbd-a3378739a5e9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:32.206916 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:32.215258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 237/466] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:32.215925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c20e7-98b0-4247-8e57-3cd7ac17ad56 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:32.309659 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:32.314164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22eeac-15b7-4c51-acbd-a3378739a5e9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:32.334554 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c20e7-98b0-4247-8e57-3cd7ac17ad56 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:32.357313 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688ef73-994a-476f-8c6c-bb3ae50260ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:32.358194 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688ef73-994a-476f-8c6c-bb3ae50260ed None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:32.358616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688ef73-994a-476f-8c6c-bb3ae50260ed None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:32.359026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688ef73-994a-476f-8c6c-bb3ae50260ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:32.373592 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0688ef73-994a-476f-8c6c-bb3ae50260ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:32.374584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee0ce1-a384-4d0a-8597-2f7d16aa63d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:32.375225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0688ef73-994a-476f-8c6c-bb3ae50260ed None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:32.375792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee0ce1-a384-4d0a-8597-2f7d16aa63d6 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:32.376247 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee0ce1-a384-4d0a-8597-2f7d16aa63d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:32.376688 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee0ce1-a384-4d0a-8597-2f7d16aa63d6 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:32.377682 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee0ce1-a384-4d0a-8597-2f7d16aa63d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:32.379632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee0ce1-a384-4d0a-8597-2f7d16aa63d6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:32.391143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-0688ef73-994a-476f-8c6c-bb3ae50260ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:32.391143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:32.391143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:32.391143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:32.391143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:32.391143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:32.391143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:32.391143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:32.391143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:32.391143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:32.391143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:32.391143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:32.391143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:32.391143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:32.391143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:32.391143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:32.392265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:32.393349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:32.393349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:32.393349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:32.399093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 230/467] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:32.400612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688ef73-994a-476f-8c6c-bb3ae50260ed None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:32.412055 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 238/468] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:08:32 2022] POST /identity/v3/users => generated 246 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:08:32.413228 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee0ce1-a384-4d0a-8597-2f7d16aa63d6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:32.479911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:32.520414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688ef73-994a-476f-8c6c-bb3ae50260ed None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:32.543056 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee0ce1-a384-4d0a-8597-2f7d16aa63d6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:32.562988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d3fcc-a3b4-45d4-a536-d28cda3857cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:32.563648 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d3fcc-a3b4-45d4-a536-d28cda3857cc None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:32.563929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d3fcc-a3b4-45d4-a536-d28cda3857cc None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:32.564202 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d3fcc-a3b4-45d4-a536-d28cda3857cc None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:32.564859 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d3fcc-a3b4-45d4-a536-d28cda3857cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:32.565160 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d3fcc-a3b4-45d4-a536-d28cda3857cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:32.566624 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d3fcc-a3b4-45d4-a536-d28cda3857cc None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:32.589347 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2140a-0393-4bac-b46e-df0ede91d21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:32.590372 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2140a-0393-4bac-b46e-df0ede91d21a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:32.590716 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 231/469] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:32.591099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2140a-0393-4bac-b46e-df0ede91d21a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:32.591600 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2140a-0393-4bac-b46e-df0ede91d21a None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:32.591771 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d3fcc-a3b4-45d4-a536-d28cda3857cc None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:32.592736 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2140a-0393-4bac-b46e-df0ede91d21a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:32.593242 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2140a-0393-4bac-b46e-df0ede91d21a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:32.594981 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2140a-0393-4bac-b46e-df0ede91d21a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:32.606662 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:32.612426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 239/470] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:32.614367 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2140a-0393-4bac-b46e-df0ede91d21a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:32.706917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d3fcc-a3b4-45d4-a536-d28cda3857cc None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:32.739636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2140a-0393-4bac-b46e-df0ede91d21a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:32.752868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33ed93-16a9-4c6f-9cbd-d414456410d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:32.753662 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33ed93-16a9-4c6f-9cbd-d414456410d6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:32.754042 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33ed93-16a9-4c6f-9cbd-d414456410d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:32.754407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33ed93-16a9-4c6f-9cbd-d414456410d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:32.756938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:32.768364 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33ed93-16a9-4c6f-9cbd-d414456410d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:32.769187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33ed93-16a9-4c6f-9cbd-d414456410d6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:32.786627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-6a33ed93-16a9-4c6f-9cbd-d414456410d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:32.786627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:32.786627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:32.786627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:32.786627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:32.786627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:32.786627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:32.786627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:32.786627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:32.786627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:32.786627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:32.786627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:32.786627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:32.786627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:32.786627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:32.786627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:32.787616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:32.788475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:32.788475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:32.788475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:32.788635 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a0a09-8c21-46e3-aa17-06f463532aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:32.789171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a0a09-8c21-46e3-aa17-06f463532aac None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:32.789414 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a0a09-8c21-46e3-aa17-06f463532aac None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:32.789777 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a0a09-8c21-46e3-aa17-06f463532aac None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:32.791018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a0a09-8c21-46e3-aa17-06f463532aac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:32.791948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 232/471] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:32.792964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33ed93-16a9-4c6f-9cbd-d414456410d6 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:32.796294 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a0a09-8c21-46e3-aa17-06f463532aac None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:32.823324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 240/472] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:08:32 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:08:32.824544 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a0a09-8c21-46e3-aa17-06f463532aac None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:32.875107 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:32.932864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33ed93-16a9-4c6f-9cbd-d414456410d6 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:32.958574 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a0a09-8c21-46e3-aa17-06f463532aac None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:32.970147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366eace-5c65-4abc-a155-c6096391db99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:32.970764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366eace-5c65-4abc-a155-c6096391db99 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:32.971051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366eace-5c65-4abc-a155-c6096391db99 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:32.971325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366eace-5c65-4abc-a155-c6096391db99 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:32.972080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366eace-5c65-4abc-a155-c6096391db99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:32.973592 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366eace-5c65-4abc-a155-c6096391db99 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:32.997828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 233/473] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:08:32 2022] POST /identity/v3/users => generated 248 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:08:32.998955 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258bf7f-5fd9-4ea9-951a-cfaa8f1582df None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:32.999247 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258bf7f-5fd9-4ea9-951a-cfaa8f1582df None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:32.999555 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258bf7f-5fd9-4ea9-951a-cfaa8f1582df None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:33.004536 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 234/474] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:33.007199 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b4aea-7195-40b7-a9c5-94ba2201771c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:33.007470 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b4aea-7195-40b7-a9c5-94ba2201771c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:33.007772 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b4aea-7195-40b7-a9c5-94ba2201771c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:33.009133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02dc22-4208-4b45-a287-1e56e3f12c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:33.010047 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02dc22-4208-4b45-a287-1e56e3f12c59 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:33.010651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02dc22-4208-4b45-a287-1e56e3f12c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:33.011388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02dc22-4208-4b45-a287-1e56e3f12c59 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:33.012416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02dc22-4208-4b45-a287-1e56e3f12c59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:33.012938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02dc22-4208-4b45-a287-1e56e3f12c59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:33.014605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02dc22-4208-4b45-a287-1e56e3f12c59 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:33.028156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-813b4aea-7195-40b7-a9c5-94ba2201771c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:33.029433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-813b4aea-7195-40b7-a9c5-94ba2201771c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:33.033977 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 241/475] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:33.040904 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02dc22-4208-4b45-a287-1e56e3f12c59 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:33.084378 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:33.113216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 235/476] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:08:33.179217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02dc22-4208-4b45-a287-1e56e3f12c59 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:33.220616 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c194bd-a169-40fe-8012-6007cb646ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:33.221597 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c194bd-a169-40fe-8012-6007cb646ab2 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:33.222088 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c194bd-a169-40fe-8012-6007cb646ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:33.222609 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c194bd-a169-40fe-8012-6007cb646ab2 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:33.223530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c194bd-a169-40fe-8012-6007cb646ab2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:33.225197 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c194bd-a169-40fe-8012-6007cb646ab2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:33.255426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 242/477] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:08:33 2022] POST /identity/v3/users => generated 249 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:08:33.426438 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b4aea-7195-40b7-a9c5-94ba2201771c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:33.479887 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:33.533897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b4aea-7195-40b7-a9c5-94ba2201771c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:33.568722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b82f3-d0b5-42c6-a838-59c4bb61161e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:33.569332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b82f3-d0b5-42c6-a838-59c4bb61161e None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:33.569609 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b82f3-d0b5-42c6-a838-59c4bb61161e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:33.569874 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b82f3-d0b5-42c6-a838-59c4bb61161e None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:33.570543 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b82f3-d0b5-42c6-a838-59c4bb61161e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:33.572172 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b82f3-d0b5-42c6-a838-59c4bb61161e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:33.583638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.sql.core [None req-d35b82f3-d0b5-42c6-a838-59c4bb61161e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 06 12:08:33.583638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 06 12:08:33.583638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [parameters: {'id': 'c34e6fcb39eb4dfd9c5c63907ae11a92', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 06 12:08:33.583638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81354) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 06 12:08:33.600999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-d35b82f3-d0b5-42c6-a838-59c4bb61161e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 06 12:08:33.600999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:33.600999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 12:08:33.600999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 12:08:33.600999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 12:08:33.600999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 12:08:33.600999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 12:08:33.600999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 12:08:33.600999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 12:08:33.600999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application conn.query(q) Oct 06 12:08:33.600999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 12:08:33.600999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 12:08:33.600999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 12:08:33.600999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result.read() Oct 06 12:08:33.600999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return ref.to_dict() Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 12:08:33.601920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application next(self.gen) Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application yield resource Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application next(self.gen) Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.session.rollback() Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.force_reraise() Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise self.value Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application session.commit() Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 06 12:08:33.604054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._prepare_impl() Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.session.flush() Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._flush(objects) Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application compat.raise_( Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application flush_context.execute() Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rec.execute(self) Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 06 12:08:33.604942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 06 12:08:33.605865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 06 12:08:33.605865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 06 12:08:33.605865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 06 12:08:33.605865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 06 12:08:33.605865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 06 12:08:33.605865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 06 12:08:33.605865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 06 12:08:33.605865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 06 12:08:33.605865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 06 12:08:33.605865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 06 12:08:33.605865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 06 12:08:33.605865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 06 12:08:33.605865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 06 12:08:33.605865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 06 12:08:33.605865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 12:08:33.605865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception Oct 06 12:08:33.606673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 12:08:33.606673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 12:08:33.606673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 12:08:33.606673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 12:08:33.606673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 12:08:33.606673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 12:08:33.606673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 12:08:33.606673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application conn.query(q) Oct 06 12:08:33.606673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 12:08:33.606673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 12:08:33.606673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 12:08:33.606673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application result.read() Oct 06 12:08:33.606673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 12:08:33.606673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 12:08:33.606673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 12:08:33.606673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 12:08:33.609417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 12:08:33.609417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 12:08:33.609417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 12:08:33.609417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 12:08:33.609417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 06 12:08:33.609417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 06 12:08:33.609417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [parameters: {'id': 'c34e6fcb39eb4dfd9c5c63907ae11a92', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 06 12:08:33.609417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 06 12:08:33.609417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:33.609417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:08:33.609417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:33.609417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:33.609417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:33.609417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:33.609417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:33.609417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:33.611372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:33.611372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:33.611372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:33.611372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:33.611372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:33.611372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:33.611372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 06 12:08:33.611372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 06 12:08:33.611372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:33.611372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:33.611372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 06 12:08:33.611372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 06 12:08:33.611372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 06 12:08:33.611372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 06 12:08:33.611372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 06 12:08:33.611372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:33.613378 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 236/478] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Oct 6 12:08:33 2022] POST /identity/v3/roles => generated 144 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:33.613514 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c194bd-a169-40fe-8012-6007cb646ab2 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:33.701341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:33.718126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c194bd-a169-40fe-8012-6007cb646ab2 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:33.753548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada8ca4-dbfe-4357-bb75-580595ca2590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:33.754113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada8ca4-dbfe-4357-bb75-580595ca2590 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:33.754392 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada8ca4-dbfe-4357-bb75-580595ca2590 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:33.754669 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada8ca4-dbfe-4357-bb75-580595ca2590 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:33.765794 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada8ca4-dbfe-4357-bb75-580595ca2590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:33.771092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-aada8ca4-dbfe-4357-bb75-580595ca2590 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:08:33.771092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:33.771092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:33.771092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:33.771092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:33.771092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:33.771092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:33.771092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:33.771092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:33.771092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:33.771092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:33.771092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:33.771092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:33.771092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:33.771092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:33.771092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:33.773292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:33.773292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:33.773292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:33.773292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:33.773292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:33.773292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:33.773292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:33.773292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:33.773292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:33.773292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:33.773292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:33.773292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:33.773292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:33.773292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:33.773292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:33.773292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:33.775381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:33.775381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:33.775381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:33.775381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:33.775381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:33.775381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:33.775381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:33.775381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:33.775381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:33.775381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:33.775381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 12:08:33.775381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:33.775381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 243/479] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:08:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:33.781221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b82f3-d0b5-42c6-a838-59c4bb61161e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:33.881900 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b82f3-d0b5-42c6-a838-59c4bb61161e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:33.895693 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:33.918853 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82fe45-e834-41ee-84dc-0ffb4f8cb472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:33.919530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82fe45-e834-41ee-84dc-0ffb4f8cb472 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:33.919820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82fe45-e834-41ee-84dc-0ffb4f8cb472 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:33.920090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82fe45-e834-41ee-84dc-0ffb4f8cb472 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:33.920751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82fe45-e834-41ee-84dc-0ffb4f8cb472 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:33.921050 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82fe45-e834-41ee-84dc-0ffb4f8cb472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:33.922409 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82fe45-e834-41ee-84dc-0ffb4f8cb472 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:33.931229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 237/480] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:35.084017 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f926f8d-0785-401d-ab80-0c08ad9ac91d None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:35.084541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f926f8d-0785-401d-ab80-0c08ad9ac91d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:35.084541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f926f8d-0785-401d-ab80-0c08ad9ac91d None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:35.086102 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 244/481] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:35.097004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92409efe-3f79-4e0b-9205-1248f46e3aad None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:35.097440 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92409efe-3f79-4e0b-9205-1248f46e3aad None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:35.097619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92409efe-3f79-4e0b-9205-1248f46e3aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:35.131483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-92409efe-3f79-4e0b-9205-1248f46e3aad None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:35.132704 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-92409efe-3f79-4e0b-9205-1248f46e3aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:35.220279 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 238/482] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:08:35.532793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f926f8d-0785-401d-ab80-0c08ad9ac91d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:35.668175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f926f8d-0785-401d-ab80-0c08ad9ac91d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:35.693575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31e1e2-6f1b-4531-b6a0-8b7cdddc25d2 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:35.693911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31e1e2-6f1b-4531-b6a0-8b7cdddc25d2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:35.694138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31e1e2-6f1b-4531-b6a0-8b7cdddc25d2 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:35.695866 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 239/483] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:35.703529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb52010-edab-4d7b-9fcf-3d18f675ce98 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:35.703529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb52010-edab-4d7b-9fcf-3d18f675ce98 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:35.703529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb52010-edab-4d7b-9fcf-3d18f675ce98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:35.708743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52464a92-fcc5-4d5d-9738-4000625b7f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:35.709382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52464a92-fcc5-4d5d-9738-4000625b7f4b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:35.709669 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52464a92-fcc5-4d5d-9738-4000625b7f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:35.710018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52464a92-fcc5-4d5d-9738-4000625b7f4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:35.724903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52464a92-fcc5-4d5d-9738-4000625b7f4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:35.727060 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8fb52010-edab-4d7b-9fcf-3d18f675ce98 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:35.728274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8fb52010-edab-4d7b-9fcf-3d18f675ce98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:35.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-52464a92-fcc5-4d5d-9738-4000625b7f4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:35.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:35.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:35.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:35.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:35.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:35.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:35.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:35.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:35.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:35.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:35.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:35.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:35.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:35.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:35.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:35.731082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:35.731082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:35.731082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:35.731082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:35.731082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:35.731082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:35.731082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:35.731082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:35.731082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:35.731082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:35.731082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:35.731082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:35.731082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:35.731082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:35.731082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:35.731082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:35.732509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:35.732509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:35.732509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:35.732509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:35.732509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:35.732509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:35.732509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:35.732509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:35.732509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:35.732509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:35.732509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:35.732509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:35.732509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 245/484] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:35.733203 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54459c9-3eaf-4e1d-8add-f949a23d301a None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:35.733269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54459c9-3eaf-4e1d-8add-f949a23d301a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:35.733519 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54459c9-3eaf-4e1d-8add-f949a23d301a None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:35.734969 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 246/485] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:35.740400 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54459c9-3eaf-4e1d-8add-f949a23d301a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:35.763864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:35.804210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 240/486] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:35.805465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa68b92-c8b8-4231-8379-fe26f5b792e7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:35.805753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa68b92-c8b8-4231-8379-fe26f5b792e7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:35.806023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa68b92-c8b8-4231-8379-fe26f5b792e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:35.824252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-caa68b92-c8b8-4231-8379-fe26f5b792e7 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:35.825422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-caa68b92-c8b8-4231-8379-fe26f5b792e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:35.853568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:35.862832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54459c9-3eaf-4e1d-8add-f949a23d301a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:35.896155 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 241/487] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:35.910619 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcc36e-1d71-4f51-b9fe-411a1e1adcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:35.911241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcc36e-1d71-4f51-b9fe-411a1e1adcc0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:35.911538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcc36e-1d71-4f51-b9fe-411a1e1adcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:35.911826 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcc36e-1d71-4f51-b9fe-411a1e1adcc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:35.912492 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcc36e-1d71-4f51-b9fe-411a1e1adcc0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:35.912795 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcc36e-1d71-4f51-b9fe-411a1e1adcc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:35.914153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcc36e-1d71-4f51-b9fe-411a1e1adcc0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:35.926773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 247/488] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:08:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:35.932590 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa68b92-c8b8-4231-8379-fe26f5b792e7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:36.078253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa68b92-c8b8-4231-8379-fe26f5b792e7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:36.126631 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15a6db-5299-4b2d-83ca-9082f13bdb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:36.127254 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15a6db-5299-4b2d-83ca-9082f13bdb5a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:36.127589 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15a6db-5299-4b2d-83ca-9082f13bdb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:36.127820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15a6db-5299-4b2d-83ca-9082f13bdb5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:36.132687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcc36e-1d71-4f51-b9fe-411a1e1adcc0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:36.145607 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15a6db-5299-4b2d-83ca-9082f13bdb5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:36.147054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15a6db-5299-4b2d-83ca-9082f13bdb5a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:36.163183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-5c15a6db-5299-4b2d-83ca-9082f13bdb5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:36.163183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:36.163183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:36.163183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:36.163183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:36.163183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:36.163183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:36.163183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:36.163183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:36.163183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:36.163183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:36.163183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:36.163183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:36.163183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:36.163183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:36.163183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:36.164255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:36.165146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:36.165146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:36.165146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:36.167157 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 242/489] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:36.176653 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15a6db-5299-4b2d-83ca-9082f13bdb5a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:36.221666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:36.275928 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:36.287092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcc36e-1d71-4f51-b9fe-411a1e1adcc0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:36.329403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cb70b-62ba-42f8-84ff-3b1f82cb3c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:36.329403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cb70b-62ba-42f8-84ff-3b1f82cb3c5e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:36.329403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cb70b-62ba-42f8-84ff-3b1f82cb3c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:36.329403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cb70b-62ba-42f8-84ff-3b1f82cb3c5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:36.339981 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1cb70b-62ba-42f8-84ff-3b1f82cb3c5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:36.345322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-dc1cb70b-62ba-42f8-84ff-3b1f82cb3c5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:36.345322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:36.345322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:36.345322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:36.345322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:36.345322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:36.345322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:36.345322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:36.345322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:36.345322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:36.345322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:36.345322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:36.345322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:36.345322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:36.345322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:36.345322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:36.346206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:36.346206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:36.346206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:36.346206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:36.346206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:36.346206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:36.346206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:36.346206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:36.346206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:36.346206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:36.346206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:36.346206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:36.346206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:36.346206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:36.346206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:36.346206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:36.347012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:36.347012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:36.347012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:36.347012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:36.347012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:36.347012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:36.347012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:36.347012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:36.347012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:36.347012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:36.347012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:36.347012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:36.347012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 248/490] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:08:36.348334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cb70b-62ba-42f8-84ff-3b1f82cb3c5e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:36.367286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15a6db-5299-4b2d-83ca-9082f13bdb5a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:36.405138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c950f-f2d1-43b3-9824-ba0f8837bf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:36.406033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c950f-f2d1-43b3-9824-ba0f8837bf3f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:36.406318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c950f-f2d1-43b3-9824-ba0f8837bf3f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:36.406887 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c950f-f2d1-43b3-9824-ba0f8837bf3f None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:36.407229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c950f-f2d1-43b3-9824-ba0f8837bf3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:36.407551 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c950f-f2d1-43b3-9824-ba0f8837bf3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:36.408945 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c950f-f2d1-43b3-9824-ba0f8837bf3f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:36.421283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 243/491] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:36.425525 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567d88d-dbdf-4c71-9367-3404bc91bd01 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:36.426031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567d88d-dbdf-4c71-9367-3404bc91bd01 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:36.426446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567d88d-dbdf-4c71-9367-3404bc91bd01 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:36.428060 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 244/492] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:36.428966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567d88d-dbdf-4c71-9367-3404bc91bd01 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:36.466413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cb70b-62ba-42f8-84ff-3b1f82cb3c5e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:36.493482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:36.503025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98049ee3-bb1d-4088-aed1-9395778d7ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:36.503687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98049ee3-bb1d-4088-aed1-9395778d7ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:36.503964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98049ee3-bb1d-4088-aed1-9395778d7ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:36.504234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98049ee3-bb1d-4088-aed1-9395778d7ad1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:36.513521 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:36.516797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98049ee3-bb1d-4088-aed1-9395778d7ad1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:36.522679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-98049ee3-bb1d-4088-aed1-9395778d7ad1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:36.522679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:36.522679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:36.522679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:36.522679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:36.522679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:36.522679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:36.522679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:36.522679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:36.522679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:36.522679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:36.522679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:36.522679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:36.522679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:36.522679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:36.522679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:36.523627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:36.523627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:36.523627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:36.523627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:36.523627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:36.523627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:36.523627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:36.523627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:36.523627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:36.523627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:36.523627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:36.523627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:36.523627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:36.523627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:36.523627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:36.523627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:36.530732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:36.530732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:36.530732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:36.530732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:36.530732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:36.530732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:36.530732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:36.530732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:36.530732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:36.530732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:36.530732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:36.530732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:36.530732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 249/493] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:36.530732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98049ee3-bb1d-4088-aed1-9395778d7ad1 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:36.544648 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567d88d-dbdf-4c71-9367-3404bc91bd01 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:36.581869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c5265-7fab-4c9e-bde3-b925b12da50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:36.582415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c5265-7fab-4c9e-bde3-b925b12da50c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:36.582695 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c5265-7fab-4c9e-bde3-b925b12da50c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:36.582969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c5265-7fab-4c9e-bde3-b925b12da50c None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:36.584083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c5265-7fab-4c9e-bde3-b925b12da50c None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:36.584398 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c5265-7fab-4c9e-bde3-b925b12da50c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:36.585764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c5265-7fab-4c9e-bde3-b925b12da50c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:36.594852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 245/494] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:08:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:36.595828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d483c-fc0e-4236-9773-7083c446737b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:36.596112 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d483c-fc0e-4236-9773-7083c446737b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:36.596386 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d483c-fc0e-4236-9773-7083c446737b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:36.613828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-284d483c-fc0e-4236-9773-7083c446737b None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:36.615013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-284d483c-fc0e-4236-9773-7083c446737b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:36.636932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98049ee3-bb1d-4088-aed1-9395778d7ad1 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:36.673497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00e190-939c-44c3-bee3-012af54ec1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:36.674124 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00e190-939c-44c3-bee3-012af54ec1dd None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:36.674400 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00e190-939c-44c3-bee3-012af54ec1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:36.674671 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00e190-939c-44c3-bee3-012af54ec1dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:36.691548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00e190-939c-44c3-bee3-012af54ec1dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:36.691708 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 246/495] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:36.691708 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d483c-fc0e-4236-9773-7083c446737b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:36.691872 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00e190-939c-44c3-bee3-012af54ec1dd None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:36.711098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-ab00e190-939c-44c3-bee3-012af54ec1dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:08:36.711098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:36.711098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:36.711098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:36.711098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:36.711098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:36.711098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:36.711098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:36.711098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:36.711098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:36.711098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:36.711098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:36.711098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:08:36.711098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:08:36.711098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:08:36.711098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:36.712651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:36.713558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:36.713558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:36.713558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:08:36.713558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:08:36.713558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:08:36.713558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:08:36.713558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:08:36.713558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:08:36.713558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:08:36.713558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:36.713558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 250/496] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Oct 6 12:08:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:36.714963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00e190-939c-44c3-bee3-012af54ec1dd None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:36.763896 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:36.766769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:36.812390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d483c-fc0e-4236-9773-7083c446737b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:36.830364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00e190-939c-44c3-bee3-012af54ec1dd None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:36.857106 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4277628-ec9e-45fc-8e7d-75aab5c30430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:36.857972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4277628-ec9e-45fc-8e7d-75aab5c30430 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:36.858258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4277628-ec9e-45fc-8e7d-75aab5c30430 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:36.858952 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4277628-ec9e-45fc-8e7d-75aab5c30430 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:36.859919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4277628-ec9e-45fc-8e7d-75aab5c30430 None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:36.860230 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4277628-ec9e-45fc-8e7d-75aab5c30430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:36.862303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4277628-ec9e-45fc-8e7d-75aab5c30430 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:36.870403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd21f7-93af-4082-b180-86ab1e9c09ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:36.871027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd21f7-93af-4082-b180-86ab1e9c09ed None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:36.871304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd21f7-93af-4082-b180-86ab1e9c09ed None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:36.871600 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd21f7-93af-4082-b180-86ab1e9c09ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:36.876450 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 247/497] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:08:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:36.877331 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4277628-ec9e-45fc-8e7d-75aab5c30430 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:36.887413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd21f7-93af-4082-b180-86ab1e9c09ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:36.888912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd21f7-93af-4082-b180-86ab1e9c09ed None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:36.908638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-84cd21f7-93af-4082-b180-86ab1e9c09ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:36.908638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:36.908638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:36.908638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:36.908638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:36.908638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:36.908638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:36.908638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:36.908638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:36.908638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:36.908638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:36.908638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:36.908638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:36.908638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:36.908638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:36.908638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:36.909572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:36.910437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:36.910437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:36.910437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:36.910437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 251/498] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:36.911140 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd21f7-93af-4082-b180-86ab1e9c09ed None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:36.980120 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4277628-ec9e-45fc-8e7d-75aab5c30430 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:37.007892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:37.014444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:37.014564 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618634b6-b1ea-4def-a1ae-e2b94d01402e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:37.015077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618634b6-b1ea-4def-a1ae-e2b94d01402e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:37.015738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618634b6-b1ea-4def-a1ae-e2b94d01402e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:37.015738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618634b6-b1ea-4def-a1ae-e2b94d01402e None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:37.016524 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618634b6-b1ea-4def-a1ae-e2b94d01402e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:37.016836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618634b6-b1ea-4def-a1ae-e2b94d01402e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:37.018255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618634b6-b1ea-4def-a1ae-e2b94d01402e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:37.018553 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd21f7-93af-4082-b180-86ab1e9c09ed None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:37.053643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e0180-06b3-43ee-a03b-c447be81a3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:37.053839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 248/499] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Oct 6 12:08:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:37.054265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e0180-06b3-43ee-a03b-c447be81a3df None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:37.054544 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e0180-06b3-43ee-a03b-c447be81a3df None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:37.054811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e0180-06b3-43ee-a03b-c447be81a3df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:37.054903 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ea203-e688-4aaa-b897-ec9d8b847021 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:37.055730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ea203-e688-4aaa-b897-ec9d8b847021 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:37.056007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ea203-e688-4aaa-b897-ec9d8b847021 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:37.057200 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 249/500] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:37.057890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ea203-e688-4aaa-b897-ec9d8b847021 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:37.069933 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e0180-06b3-43ee-a03b-c447be81a3df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:37.072050 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e0180-06b3-43ee-a03b-c447be81a3df None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:37.086419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-b79e0180-06b3-43ee-a03b-c447be81a3df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:37.086419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:37.086419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:37.086419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:37.086419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:37.086419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:37.086419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:37.086419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:37.086419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:37.086419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:37.086419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:37.086419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:37.086419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:37.086419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:37.086419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:37.086419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:37.087338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:37.088239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:37.088239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:37.088239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:37.088239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 252/501] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:37.088463 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e0180-06b3-43ee-a03b-c447be81a3df None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:37.162596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ea203-e688-4aaa-b897-ec9d8b847021 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:37.191198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e0180-06b3-43ee-a03b-c447be81a3df None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:37.195242 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e04749-790f-4f08-9924-63fa82dbf892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:37.195953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e04749-790f-4f08-9924-63fa82dbf892 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:37.196725 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e04749-790f-4f08-9924-63fa82dbf892 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:37.196997 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e04749-790f-4f08-9924-63fa82dbf892 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:37.197633 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e04749-790f-4f08-9924-63fa82dbf892 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:37.197935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e04749-790f-4f08-9924-63fa82dbf892 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:37.199315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e04749-790f-4f08-9924-63fa82dbf892 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:37.210250 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 250/502] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:37.211035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e04749-790f-4f08-9924-63fa82dbf892 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:37.228945 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752ddfa-265c-4741-a6d4-a5fcd0d14c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:37.229570 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752ddfa-265c-4741-a6d4-a5fcd0d14c71 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:37.229853 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752ddfa-265c-4741-a6d4-a5fcd0d14c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:37.230130 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752ddfa-265c-4741-a6d4-a5fcd0d14c71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:37.240898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752ddfa-265c-4741-a6d4-a5fcd0d14c71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:37.246607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-7752ddfa-265c-4741-a6d4-a5fcd0d14c71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:37.246607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:37.246607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:37.246607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:37.246607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:37.246607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:37.246607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:37.246607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:37.246607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:37.246607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:37.246607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:37.246607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:37.246607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:37.246607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:37.246607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:37.246607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:37.247785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:37.247785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:37.247785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:37.247785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:37.247785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:37.247785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:37.247785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:37.247785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:37.247785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:37.247785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:37.247785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:37.247785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:37.247785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:37.247785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:37.247785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:37.247785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:37.249035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:37.249035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:37.249035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:37.249035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:37.249035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:37.249035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:37.249035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:37.249035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:37.249035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:37.249035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:37.249035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:37.249035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:37.249035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 253/503] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:37.249710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622549e-b41f-4a79-a92c-a92f8537568d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:37.249710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622549e-b41f-4a79-a92c-a92f8537568d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:37.249830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622549e-b41f-4a79-a92c-a92f8537568d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:37.263046 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:37.267914 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-2622549e-b41f-4a79-a92c-a92f8537568d None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:37.269111 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2622549e-b41f-4a79-a92c-a92f8537568d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:37.271856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:37.313137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e04749-790f-4f08-9924-63fa82dbf892 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:37.338404 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 254/504] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:37.339232 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622549e-b41f-4a79-a92c-a92f8537568d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:37.346243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79835d2d-175b-4c0b-a3a8-ab74dca93e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:37.346845 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79835d2d-175b-4c0b-a3a8-ab74dca93e0c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:37.347120 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79835d2d-175b-4c0b-a3a8-ab74dca93e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:37.347385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79835d2d-175b-4c0b-a3a8-ab74dca93e0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:37.415346 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79835d2d-175b-4c0b-a3a8-ab74dca93e0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:37.416809 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79835d2d-175b-4c0b-a3a8-ab74dca93e0c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:37.464275 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:37.464962 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622549e-b41f-4a79-a92c-a92f8537568d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:37.500872 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da40c2-9e4e-4039-b841-4382a3844833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:37.501520 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da40c2-9e4e-4039-b841-4382a3844833 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:37.501908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da40c2-9e4e-4039-b841-4382a3844833 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:37.502225 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da40c2-9e4e-4039-b841-4382a3844833 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:37.502922 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da40c2-9e4e-4039-b841-4382a3844833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:37.503221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da40c2-9e4e-4039-b841-4382a3844833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:37.504610 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da40c2-9e4e-4039-b841-4382a3844833 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:37.513938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:37.521808 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 255/505] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:37.521808 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da40c2-9e4e-4039-b841-4382a3844833 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:37.522536 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 251/506] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:37.523362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79835d2d-175b-4c0b-a3a8-ab74dca93e0c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:37.623873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79835d2d-175b-4c0b-a3a8-ab74dca93e0c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:37.630713 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da40c2-9e4e-4039-b841-4382a3844833 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:37.658119 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f55bf-08cc-4dca-8e62-14c1e637bc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:37.658724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f55bf-08cc-4dca-8e62-14c1e637bc85 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:37.659004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f55bf-08cc-4dca-8e62-14c1e637bc85 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:37.659276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f55bf-08cc-4dca-8e62-14c1e637bc85 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:37.659980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f55bf-08cc-4dca-8e62-14c1e637bc85 None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:37.660286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f55bf-08cc-4dca-8e62-14c1e637bc85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:37.661681 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f55bf-08cc-4dca-8e62-14c1e637bc85 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:37.666762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444058e-d186-45c4-bfe4-1c2993963676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:37.667322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444058e-d186-45c4-bfe4-1c2993963676 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:37.667629 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444058e-d186-45c4-bfe4-1c2993963676 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:37.667904 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444058e-d186-45c4-bfe4-1c2993963676 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:37.669631 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 252/507] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:08:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:37.670506 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f55bf-08cc-4dca-8e62-14c1e637bc85 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:37.678334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444058e-d186-45c4-bfe4-1c2993963676 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:37.679826 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444058e-d186-45c4-bfe4-1c2993963676 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:37.684338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:37.691894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-8444058e-d186-45c4-bfe4-1c2993963676 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 06 12:08:37.691894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:37.691894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:37.691894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:37.691894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:37.691894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:37.691894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:37.691894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:37.691894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:37.691894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:37.691894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:37.691894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:37.691894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:08:37.691894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:08:37.691894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:08:37.691894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:37.692823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:37.693734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:37.693734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:37.693734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:08:37.693734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:08:37.693734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:08:37.693734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:08:37.693734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:08:37.693734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:08:37.693734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 06 12:08:37.693734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:37.693734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 256/508] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:37.694875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444058e-d186-45c4-bfe4-1c2993963676 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:37.754595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:37.773558 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f55bf-08cc-4dca-8e62-14c1e637bc85 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:37.795931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444058e-d186-45c4-bfe4-1c2993963676 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:37.806307 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca149871-e3b4-44fa-9021-db9401803831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:37.806944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca149871-e3b4-44fa-9021-db9401803831 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:37.807222 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca149871-e3b4-44fa-9021-db9401803831 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:37.807499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca149871-e3b4-44fa-9021-db9401803831 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:37.814666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca149871-e3b4-44fa-9021-db9401803831 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:37.816158 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca149871-e3b4-44fa-9021-db9401803831 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:37.828363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-ca149871-e3b4-44fa-9021-db9401803831 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 12:08:37.828363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:37.828363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:37.828363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:37.828363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:37.828363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:37.828363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:37.828363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:37.828363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:37.828363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:37.828363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:37.828363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:37.828363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:08:37.828363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:08:37.828363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:08:37.828363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:37.829402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:37.830314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:37.830314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:37.830314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:08:37.830314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:08:37.830314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:08:37.830314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:08:37.830314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:08:37.830314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:08:37.830314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 12:08:37.830314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:37.830826 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cbf0e-7feb-4a61-badf-de94dd41dbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:37.830918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 253/509] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:37.830986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cbf0e-7feb-4a61-badf-de94dd41dbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:37.830986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cbf0e-7feb-4a61-badf-de94dd41dbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:37.830986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cbf0e-7feb-4a61-badf-de94dd41dbe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:37.831154 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca149871-e3b4-44fa-9021-db9401803831 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:37.844827 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cbf0e-7feb-4a61-badf-de94dd41dbe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:37.846278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cbf0e-7feb-4a61-badf-de94dd41dbe9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:37.860255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-a29cbf0e-7feb-4a61-badf-de94dd41dbe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:37.860255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:37.860255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:37.860255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:37.860255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:37.860255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:37.860255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:37.860255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:37.860255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:37.860255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:37.860255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:37.860255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:37.860255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:37.860255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:37.860255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:37.860255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:37.861134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:37.862022 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:37.862022 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:37.862022 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:37.862022 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 257/510] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:37.862244 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cbf0e-7feb-4a61-badf-de94dd41dbe9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:37.906033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:37.937216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca149871-e3b4-44fa-9021-db9401803831 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:37.959659 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cbf0e-7feb-4a61-badf-de94dd41dbe9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:37.969987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1282b-18fc-4b0e-ae41-1626f27def08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:37.970613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1282b-18fc-4b0e-ae41-1626f27def08 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:37.970896 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1282b-18fc-4b0e-ae41-1626f27def08 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:37.971173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1282b-18fc-4b0e-ae41-1626f27def08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:37.972540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:37.982096 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1282b-18fc-4b0e-ae41-1626f27def08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:37.983588 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1282b-18fc-4b0e-ae41-1626f27def08 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:37.993097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96a729-86ee-4494-b4d7-39b7265a508f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:37.993710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96a729-86ee-4494-b4d7-39b7265a508f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:37.993988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96a729-86ee-4494-b4d7-39b7265a508f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:37.994254 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96a729-86ee-4494-b4d7-39b7265a508f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:37.998051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-e9f1282b-18fc-4b0e-ae41-1626f27def08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:37.998051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:37.998051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:37.998051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:37.998051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:37.998051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:37.998051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:37.998051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:37.998051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:37.998051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:37.998051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:37.998051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:37.998051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:37.998051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:37.998051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:37.998051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:37.999138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:38.000139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:38.000139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:38.000139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:38.000139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 254/511] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:38.000416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1282b-18fc-4b0e-ae41-1626f27def08 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:38.005847 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96a729-86ee-4494-b4d7-39b7265a508f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:38.007398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96a729-86ee-4494-b4d7-39b7265a508f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:38.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-3e96a729-86ee-4494-b4d7-39b7265a508f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:38.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:38.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:38.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:38.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:38.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:38.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:38.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:38.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:38.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:38.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:38.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:38.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:38.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:38.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:38.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:38.022667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:38.023576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:38.023576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:38.023576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:38.023576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 258/512] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:38.024332 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96a729-86ee-4494-b4d7-39b7265a508f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:38.103345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1282b-18fc-4b0e-ae41-1626f27def08 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:38.122318 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96a729-86ee-4494-b4d7-39b7265a508f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:38.124815 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:38.137160 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad583c6-ee8d-4cb1-a787-504007f8530c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:38.137766 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad583c6-ee8d-4cb1-a787-504007f8530c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:38.138048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad583c6-ee8d-4cb1-a787-504007f8530c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:38.138324 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad583c6-ee8d-4cb1-a787-504007f8530c None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:38.139046 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad583c6-ee8d-4cb1-a787-504007f8530c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:38.139351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad583c6-ee8d-4cb1-a787-504007f8530c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:38.140827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad583c6-ee8d-4cb1-a787-504007f8530c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:38.156653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f6ff8-b852-4a34-b6a9-ef4151f6a6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:38.157254 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f6ff8-b852-4a34-b6a9-ef4151f6a6b6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:38.157535 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f6ff8-b852-4a34-b6a9-ef4151f6a6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:38.158059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f6ff8-b852-4a34-b6a9-ef4151f6a6b6 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:38.158157 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 255/513] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Oct 6 12:08:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:38.158523 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f6ff8-b852-4a34-b6a9-ef4151f6a6b6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:38.158762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad583c6-ee8d-4cb1-a787-504007f8530c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:38.158845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f6ff8-b852-4a34-b6a9-ef4151f6a6b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:38.160273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f6ff8-b852-4a34-b6a9-ef4151f6a6b6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:38.197676 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 259/514] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Oct 6 12:08:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:38.198437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f6ff8-b852-4a34-b6a9-ef4151f6a6b6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:38.223113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:38.260788 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad583c6-ee8d-4cb1-a787-504007f8530c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:38.294445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92831b3-9dfb-4428-82c5-3e189dcb92a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:38.295049 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92831b3-9dfb-4428-82c5-3e189dcb92a2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:38.295334 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92831b3-9dfb-4428-82c5-3e189dcb92a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:38.295647 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92831b3-9dfb-4428-82c5-3e189dcb92a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:38.296281 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92831b3-9dfb-4428-82c5-3e189dcb92a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:38.296585 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92831b3-9dfb-4428-82c5-3e189dcb92a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:38.297974 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92831b3-9dfb-4428-82c5-3e189dcb92a2 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:38.306505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f6ff8-b852-4a34-b6a9-ef4151f6a6b6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:38.308936 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 256/515] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:38.309715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92831b3-9dfb-4428-82c5-3e189dcb92a2 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:38.343016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33191de3-3d4d-445b-9532-7d586e15c227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:38.344195 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33191de3-3d4d-445b-9532-7d586e15c227 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:38.344481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33191de3-3d4d-445b-9532-7d586e15c227 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:38.344756 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33191de3-3d4d-445b-9532-7d586e15c227 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:38.345396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33191de3-3d4d-445b-9532-7d586e15c227 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:38.345703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33191de3-3d4d-445b-9532-7d586e15c227 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:38.347081 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33191de3-3d4d-445b-9532-7d586e15c227 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:38.349577 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:38.358904 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 260/516] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:38.359730 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33191de3-3d4d-445b-9532-7d586e15c227 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:38.409940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92831b3-9dfb-4428-82c5-3e189dcb92a2 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:38.439882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:38.442502 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e86c7-aefc-4e12-9015-24aea73a1efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:38.443108 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e86c7-aefc-4e12-9015-24aea73a1efc None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:38.443390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e86c7-aefc-4e12-9015-24aea73a1efc None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:38.443696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e86c7-aefc-4e12-9015-24aea73a1efc None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:38.444331 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e86c7-aefc-4e12-9015-24aea73a1efc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:38.444633 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e86c7-aefc-4e12-9015-24aea73a1efc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:38.445996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e86c7-aefc-4e12-9015-24aea73a1efc None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:38.457438 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 257/517] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:38.457887 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33191de3-3d4d-445b-9532-7d586e15c227 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:38.458158 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e86c7-aefc-4e12-9015-24aea73a1efc None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:38.494243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ad3dd-1c13-477a-b2c9-01dc15312596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:38.494830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ad3dd-1c13-477a-b2c9-01dc15312596 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:38.495106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ad3dd-1c13-477a-b2c9-01dc15312596 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:38.495376 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ad3dd-1c13-477a-b2c9-01dc15312596 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:38.551723 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:38.555474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e86c7-aefc-4e12-9015-24aea73a1efc None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:38.562741 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ad3dd-1c13-477a-b2c9-01dc15312596 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:38.564258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ad3dd-1c13-477a-b2c9-01dc15312596 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:38.589361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa06bf-b1ce-4813-9769-ef2e7e4477fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:38.589963 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa06bf-b1ce-4813-9769-ef2e7e4477fc None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:38.590244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa06bf-b1ce-4813-9769-ef2e7e4477fc None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:38.590522 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa06bf-b1ce-4813-9769-ef2e7e4477fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:38.643974 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:38.656132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa06bf-b1ce-4813-9769-ef2e7e4477fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:38.657577 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa06bf-b1ce-4813-9769-ef2e7e4477fc None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:38.664132 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 261/518] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:38.664847 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ad3dd-1c13-477a-b2c9-01dc15312596 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:38.751414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:38.755072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 258/519] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:38.756143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa06bf-b1ce-4813-9769-ef2e7e4477fc None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:38.765900 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ad3dd-1c13-477a-b2c9-01dc15312596 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:38.799885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c6864-c0be-4e49-bab4-9886311bf6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:38.800836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c6864-c0be-4e49-bab4-9886311bf6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:38.801114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c6864-c0be-4e49-bab4-9886311bf6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:38.801382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c6864-c0be-4e49-bab4-9886311bf6f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:38.809040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c6864-c0be-4e49-bab4-9886311bf6f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:38.810476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c6864-c0be-4e49-bab4-9886311bf6f2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:38.822645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-ac5c6864-c0be-4e49-bab4-9886311bf6f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:38.822645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:38.822645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:38.822645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:38.822645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:38.822645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:38.822645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:38.822645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:38.822645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:38.822645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:38.822645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:38.822645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:38.822645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:08:38.822645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:08:38.822645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:38.823491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:38.824364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:08:38.824364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:08:38.824364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:08:38.824364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:08:38.824364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:08:38.824364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:08:38.824364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:38.824364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:38.824364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 262/520] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:38.824851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c6864-c0be-4e49-bab4-9886311bf6f2 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:38.859355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa06bf-b1ce-4813-9769-ef2e7e4477fc None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:38.865719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:38.892961 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b6da3-e570-497e-9828-095448892314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:38.893566 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b6da3-e570-497e-9828-095448892314 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:38.893849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b6da3-e570-497e-9828-095448892314 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:38.894117 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b6da3-e570-497e-9828-095448892314 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:38.894754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b6da3-e570-497e-9828-095448892314 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:38.896219 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b6da3-e570-497e-9828-095448892314 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:38.917056 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 259/521] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:08:38 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:08:38.918045 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b6da3-e570-497e-9828-095448892314 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:38.923824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c6864-c0be-4e49-bab4-9886311bf6f2 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:38.963230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0739d-1cb0-485f-8026-b57e89e4c46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:38.963906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0739d-1cb0-485f-8026-b57e89e4c46a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:38.964190 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0739d-1cb0-485f-8026-b57e89e4c46a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:38.964462 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0739d-1cb0-485f-8026-b57e89e4c46a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:38.973650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a0739d-1cb0-485f-8026-b57e89e4c46a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:38.975185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a0739d-1cb0-485f-8026-b57e89e4c46a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:38.988359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-35a0739d-1cb0-485f-8026-b57e89e4c46a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 12:08:38.988359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:38.988359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:38.988359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:38.988359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:38.988359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:38.988359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:38.988359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:38.988359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:38.988359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:38.988359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:38.988359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:38.988359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:08:38.988359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:08:38.988359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:08:38.988359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:38.989362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:38.990275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:38.990275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:38.990275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:08:38.990275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:08:38.990275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:08:38.990275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:08:38.990275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:08:38.990275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:08:38.990275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 12:08:38.990275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:38.993320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:38.995291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 263/522] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:38.996419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0739d-1cb0-485f-8026-b57e89e4c46a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:39.043994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b6da3-e570-497e-9828-095448892314 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:39.079556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad8de5-562d-4be4-b824-a61a4feb150a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:39.080202 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad8de5-562d-4be4-b824-a61a4feb150a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:39.080482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad8de5-562d-4be4-b824-a61a4feb150a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:39.080745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad8de5-562d-4be4-b824-a61a4feb150a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:39.091927 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad8de5-562d-4be4-b824-a61a4feb150a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:39.093379 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad8de5-562d-4be4-b824-a61a4feb150a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:39.109346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0739d-1cb0-485f-8026-b57e89e4c46a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:39.109818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-18ad8de5-562d-4be4-b824-a61a4feb150a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:39.109818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:39.109818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:39.109818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:39.109818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:39.109818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:39.109818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:39.109818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:39.109818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:39.109818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:39.109818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:39.109818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:39.109818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:39.109818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:39.109818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:39.109818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:39.110701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:39.111732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:39.111732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:39.111732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:39.111732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 260/523] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:39.111948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:39.112026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad8de5-562d-4be4-b824-a61a4feb150a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:39.163882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d56a2-2065-4bae-9d52-9eaba2c1a272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:39.165147 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d56a2-2065-4bae-9d52-9eaba2c1a272 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:39.165520 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d56a2-2065-4bae-9d52-9eaba2c1a272 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:39.165859 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d56a2-2065-4bae-9d52-9eaba2c1a272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:39.178855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d56a2-2065-4bae-9d52-9eaba2c1a272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:39.180465 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d56a2-2065-4bae-9d52-9eaba2c1a272 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:39.194740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-938d56a2-2065-4bae-9d52-9eaba2c1a272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:39.194740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:39.194740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:39.194740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:39.194740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:39.194740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:39.194740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:39.194740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:39.194740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:39.194740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:39.194740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:39.194740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:39.194740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:39.194740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:39.194740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:39.194740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:39.195792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:39.196700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:39.196700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:39.196700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:39.196700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 264/524] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:39.197780 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d56a2-2065-4bae-9d52-9eaba2c1a272 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:39.228304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad8de5-562d-4be4-b824-a61a4feb150a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:39.237829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:39.269747 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba340b5-acd3-48cb-af21-760a557628d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:39.270350 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba340b5-acd3-48cb-af21-760a557628d1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:39.270617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba340b5-acd3-48cb-af21-760a557628d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:39.270886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba340b5-acd3-48cb-af21-760a557628d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:39.271609 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba340b5-acd3-48cb-af21-760a557628d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:39.271911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba340b5-acd3-48cb-af21-760a557628d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:39.273383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba340b5-acd3-48cb-af21-760a557628d1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:39.296469 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 261/525] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:08:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:39.296469 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba340b5-acd3-48cb-af21-760a557628d1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:39.316293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d56a2-2065-4bae-9d52-9eaba2c1a272 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:39.354388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efeaba1-d86f-4d99-b322-27e4e859a5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:39.354992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efeaba1-d86f-4d99-b322-27e4e859a5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:39.355271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efeaba1-d86f-4d99-b322-27e4e859a5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:39.355589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efeaba1-d86f-4d99-b322-27e4e859a5d2 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:39.356292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efeaba1-d86f-4d99-b322-27e4e859a5d2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:39.356598 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efeaba1-d86f-4d99-b322-27e4e859a5d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:39.358029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efeaba1-d86f-4d99-b322-27e4e859a5d2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:39.380087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 265/526] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Oct 6 12:08:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:39.380087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efeaba1-d86f-4d99-b322-27e4e859a5d2 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:39.394614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:39.401881 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba340b5-acd3-48cb-af21-760a557628d1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:39.447261 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a020b-42fe-413d-b04e-5825db26f4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:39.447901 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a020b-42fe-413d-b04e-5825db26f4de None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:39.448178 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a020b-42fe-413d-b04e-5825db26f4de None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:39.448448 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a020b-42fe-413d-b04e-5825db26f4de None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:39.449343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a020b-42fe-413d-b04e-5825db26f4de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:39.449645 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a020b-42fe-413d-b04e-5825db26f4de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:39.451948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a020b-42fe-413d-b04e-5825db26f4de None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:39.468235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 262/527] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:39.468235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a020b-42fe-413d-b04e-5825db26f4de None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:39.491989 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:39.496312 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efeaba1-d86f-4d99-b322-27e4e859a5d2 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:39.532953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0636a1-2720-4711-ae63-baa66119263c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:39.533562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0636a1-2720-4711-ae63-baa66119263c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:39.533845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0636a1-2720-4711-ae63-baa66119263c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:39.534118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0636a1-2720-4711-ae63-baa66119263c None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:39.534752 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0636a1-2720-4711-ae63-baa66119263c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:39.535063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0636a1-2720-4711-ae63-baa66119263c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:39.536451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0636a1-2720-4711-ae63-baa66119263c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:39.553079 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 266/528] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:39.553079 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0636a1-2720-4711-ae63-baa66119263c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:39.569506 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a020b-42fe-413d-b04e-5825db26f4de None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:39.604385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b14e6-0630-48d4-90ce-14df7678cf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:39.605108 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b14e6-0630-48d4-90ce-14df7678cf12 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:39.605397 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b14e6-0630-48d4-90ce-14df7678cf12 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:39.605671 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b14e6-0630-48d4-90ce-14df7678cf12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:39.606402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b14e6-0630-48d4-90ce-14df7678cf12 None admin] RBAC: Adding query filter params (role.id=910821ade2904d23b75fa69e040fb82d, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=5a46e63d34564d34b6b5752706375798) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:39.606711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b14e6-0630-48d4-90ce-14df7678cf12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:39.608336 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b14e6-0630-48d4-90ce-14df7678cf12 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:39.623938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 263/529] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:08:39 2022] GET /identity/v3/role_assignments?user.id=5a46e63d34564d34b6b5752706375798&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=910821ade2904d23b75fa69e040fb82d => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:39.623938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b14e6-0630-48d4-90ce-14df7678cf12 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:39.642647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:39.677378 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0636a1-2720-4711-ae63-baa66119263c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:39.721769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760a500-d4a8-47ae-b9ab-beb158362ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:39.723088 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760a500-d4a8-47ae-b9ab-beb158362ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:39.723386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760a500-d4a8-47ae-b9ab-beb158362ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:39.723698 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760a500-d4a8-47ae-b9ab-beb158362ba4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:39.752258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:39.763710 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b14e6-0630-48d4-90ce-14df7678cf12 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:39.811669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3975dc1-9ae6-4679-a1f1-2360853d6eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:39.812289 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3975dc1-9ae6-4679-a1f1-2360853d6eac None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:39.812576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3975dc1-9ae6-4679-a1f1-2360853d6eac None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:39.812857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3975dc1-9ae6-4679-a1f1-2360853d6eac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:39.822992 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3975dc1-9ae6-4679-a1f1-2360853d6eac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:39.823561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3760a500-d4a8-47ae-b9ab-beb158362ba4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:39.827734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3975dc1-9ae6-4679-a1f1-2360853d6eac None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:39.827869 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3760a500-d4a8-47ae-b9ab-beb158362ba4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:39.839914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-f3975dc1-9ae6-4679-a1f1-2360853d6eac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:39.839914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:39.839914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:39.839914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:39.839914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:39.839914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:39.839914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:39.839914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:39.839914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:39.839914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:39.839914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:39.839914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:39.839914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:08:39.839914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:08:39.839914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:39.840861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:39.841715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:08:39.841715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:08:39.841715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:08:39.841715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:08:39.841715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:08:39.841715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:08:39.841715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:39.841715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:39.841715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 264/530] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:39.843208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3975dc1-9ae6-4679-a1f1-2360853d6eac None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:39.909968 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:39.951594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 267/531] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:39.952692 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760a500-d4a8-47ae-b9ab-beb158362ba4 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:39.967835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3975dc1-9ae6-4679-a1f1-2360853d6eac None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:40.003443 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a513f-6299-4abb-8b99-e9ed51645683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:40.004791 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a513f-6299-4abb-8b99-e9ed51645683 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:40.005222 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a513f-6299-4abb-8b99-e9ed51645683 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:40.005946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a513f-6299-4abb-8b99-e9ed51645683 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:40.015233 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a513f-6299-4abb-8b99-e9ed51645683 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:40.017330 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a513f-6299-4abb-8b99-e9ed51645683 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:40.033368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-7b8a513f-6299-4abb-8b99-e9ed51645683 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:40.033368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:40.033368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:40.033368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:40.033368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:40.033368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:40.033368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:40.033368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:40.033368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:40.033368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:40.033368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:40.033368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:40.033368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:08:40.033368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:08:40.033368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:40.034399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:40.035494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:08:40.035494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:08:40.035494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:08:40.035494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:08:40.035494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:08:40.035494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:08:40.035494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:40.035494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:40.036701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 265/532] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:40.038354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a513f-6299-4abb-8b99-e9ed51645683 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:40.057246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:40.076523 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760a500-d4a8-47ae-b9ab-beb158362ba4 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:40.120022 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e751ff-cf38-4165-9fc4-28e54a854e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:40.120686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e751ff-cf38-4165-9fc4-28e54a854e85 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:40.120973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e751ff-cf38-4165-9fc4-28e54a854e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:40.121254 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e751ff-cf38-4165-9fc4-28e54a854e85 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:40.122076 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e751ff-cf38-4165-9fc4-28e54a854e85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:40.122386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e751ff-cf38-4165-9fc4-28e54a854e85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:40.123975 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e751ff-cf38-4165-9fc4-28e54a854e85 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:40.145862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 268/533] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:08:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:40.146992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e751ff-cf38-4165-9fc4-28e54a854e85 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:40.168285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a513f-6299-4abb-8b99-e9ed51645683 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:40.203611 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6beab-b3e7-4205-b30e-de4d26a3b626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:40.204248 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6beab-b3e7-4205-b30e-de4d26a3b626 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:40.204528 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6beab-b3e7-4205-b30e-de4d26a3b626 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:40.204800 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6beab-b3e7-4205-b30e-de4d26a3b626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:40.216233 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6beab-b3e7-4205-b30e-de4d26a3b626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:40.217705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6beab-b3e7-4205-b30e-de4d26a3b626 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:40.232294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-1fe6beab-b3e7-4205-b30e-de4d26a3b626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:40.232294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:40.232294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:40.232294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:40.232294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:40.232294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:40.232294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:40.232294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:40.232294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:40.232294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:40.232294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:40.232294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:40.232294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:40.232294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:40.232294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:40.232294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:40.233267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:40.234133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:40.234133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:40.234133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:40.234133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 266/534] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:40.234829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6beab-b3e7-4205-b30e-de4d26a3b626 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:40.246169 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:40.296271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e751ff-cf38-4165-9fc4-28e54a854e85 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:40.315111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:40.342280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f3b44-a8e1-4fdf-b17c-adfb314aa246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:40.343255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f3b44-a8e1-4fdf-b17c-adfb314aa246 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:40.344169 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f3b44-a8e1-4fdf-b17c-adfb314aa246 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:40.344594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f3b44-a8e1-4fdf-b17c-adfb314aa246 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:40.345721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f3b44-a8e1-4fdf-b17c-adfb314aa246 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:40.346173 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f3b44-a8e1-4fdf-b17c-adfb314aa246 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:40.348018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f3b44-a8e1-4fdf-b17c-adfb314aa246 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:40.357167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6beab-b3e7-4205-b30e-de4d26a3b626 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:40.364256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 269/535] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:08:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:40.365318 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f3b44-a8e1-4fdf-b17c-adfb314aa246 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:40.401890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9c716-db2e-4290-8276-f0c6e4babb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:40.402975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9c716-db2e-4290-8276-f0c6e4babb14 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:40.403432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9c716-db2e-4290-8276-f0c6e4babb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:40.403904 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9c716-db2e-4290-8276-f0c6e4babb14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:40.404817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9c716-db2e-4290-8276-f0c6e4babb14 None admin] RBAC: Adding query filter params (role.id=910821ade2904d23b75fa69e040fb82d, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=6a889616f86346f48b7573969174ee57) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:40.405557 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9c716-db2e-4290-8276-f0c6e4babb14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:40.407549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9c716-db2e-4290-8276-f0c6e4babb14 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:40.413740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 267/536] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:08:40 2022] GET /identity/v3/role_assignments?user.id=6a889616f86346f48b7573969174ee57&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=910821ade2904d23b75fa69e040fb82d => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:40.415410 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9c716-db2e-4290-8276-f0c6e4babb14 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:40.475975 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:40.487103 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f3b44-a8e1-4fdf-b17c-adfb314aa246 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:40.527169 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac00a8-9646-4b1b-96f2-1d5bb67c2dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:40.528566 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac00a8-9646-4b1b-96f2-1d5bb67c2dac None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:40.529574 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac00a8-9646-4b1b-96f2-1d5bb67c2dac None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:40.530472 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac00a8-9646-4b1b-96f2-1d5bb67c2dac None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:40.531483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac00a8-9646-4b1b-96f2-1d5bb67c2dac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:40.532123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac00a8-9646-4b1b-96f2-1d5bb67c2dac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:40.533867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac00a8-9646-4b1b-96f2-1d5bb67c2dac None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:40.535959 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9c716-db2e-4290-8276-f0c6e4babb14 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:40.564402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 270/537] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:40.573278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac00a8-9646-4b1b-96f2-1d5bb67c2dac None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:40.601027 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:40.621946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb557d8-ebed-48a2-b3a9-679a168fd6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:40.622576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb557d8-ebed-48a2-b3a9-679a168fd6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:40.622859 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb557d8-ebed-48a2-b3a9-679a168fd6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:40.623135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb557d8-ebed-48a2-b3a9-679a168fd6e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:40.623907 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb557d8-ebed-48a2-b3a9-679a168fd6e9 None admin] RBAC: Adding query filter params (role.id=910821ade2904d23b75fa69e040fb82d, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=43e873f881e6497ea09f7f10f08e3bd4) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:40.624219 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb557d8-ebed-48a2-b3a9-679a168fd6e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:40.625683 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb557d8-ebed-48a2-b3a9-679a168fd6e9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:40.639251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 268/538] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:08:40 2022] GET /identity/v3/role_assignments?user.id=43e873f881e6497ea09f7f10f08e3bd4&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=910821ade2904d23b75fa69e040fb82d => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:40.725349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac00a8-9646-4b1b-96f2-1d5bb67c2dac None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:40.769274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa42eda-dd73-4795-a7f3-b921864b5b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:40.770312 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa42eda-dd73-4795-a7f3-b921864b5b03 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:40.770793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa42eda-dd73-4795-a7f3-b921864b5b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:40.771277 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa42eda-dd73-4795-a7f3-b921864b5b03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:40.780060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa42eda-dd73-4795-a7f3-b921864b5b03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:40.782154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa42eda-dd73-4795-a7f3-b921864b5b03 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:40.799743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-4aa42eda-dd73-4795-a7f3-b921864b5b03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:40.799743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:40.799743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:40.799743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:40.799743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:40.799743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:40.799743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:40.799743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:40.799743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:40.799743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:40.799743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:40.799743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:40.799743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:08:40.799743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:08:40.799743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:40.800687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:40.801563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:08:40.801563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:08:40.801563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:08:40.801563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:08:40.801563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:08:40.801563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:08:40.801563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:40.801563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:40.808104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 271/539] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:40.813948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb557d8-ebed-48a2-b3a9-679a168fd6e9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:40.958326 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb557d8-ebed-48a2-b3a9-679a168fd6e9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:41.003236 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a4c55-3bf1-4e1d-94fc-a67cd87cf9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:41.003869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a4c55-3bf1-4e1d-94fc-a67cd87cf9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:41.004152 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a4c55-3bf1-4e1d-94fc-a67cd87cf9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:41.004426 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a4c55-3bf1-4e1d-94fc-a67cd87cf9b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:41.005122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a4c55-3bf1-4e1d-94fc-a67cd87cf9b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:41.005428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a4c55-3bf1-4e1d-94fc-a67cd87cf9b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:41.006867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a4c55-3bf1-4e1d-94fc-a67cd87cf9b2 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:41.022683 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 269/540] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:08:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:41.029085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa42eda-dd73-4795-a7f3-b921864b5b03 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:41.071874 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:41.152810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa42eda-dd73-4795-a7f3-b921864b5b03 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:41.194992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec5664-a06d-4bd4-91b5-f766410b8dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:41.196184 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec5664-a06d-4bd4-91b5-f766410b8dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:41.196471 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec5664-a06d-4bd4-91b5-f766410b8dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:41.196743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec5664-a06d-4bd4-91b5-f766410b8dd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:41.197481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec5664-a06d-4bd4-91b5-f766410b8dd2 None admin] RBAC: Adding query filter params (role.id=910821ade2904d23b75fa69e040fb82d, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=2e71f9c1626f4f58916a5dd5c5987ccb) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:41.197789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec5664-a06d-4bd4-91b5-f766410b8dd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:41.199363 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec5664-a06d-4bd4-91b5-f766410b8dd2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:41.205544 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 272/541] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:08:41 2022] GET /identity/v3/role_assignments?user.id=2e71f9c1626f4f58916a5dd5c5987ccb&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=910821ade2904d23b75fa69e040fb82d => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:41.328719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0aff7-5bb0-4ff8-9929-02396cba6f52 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:41.329259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0aff7-5bb0-4ff8-9929-02396cba6f52 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:41.329259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0aff7-5bb0-4ff8-9929-02396cba6f52 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:41.330873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 270/542] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:41.336091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82092e42-aaea-4d70-9892-71c6847c689b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:41.336629 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82092e42-aaea-4d70-9892-71c6847c689b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:41.336708 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82092e42-aaea-4d70-9892-71c6847c689b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:41.361326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-82092e42-aaea-4d70-9892-71c6847c689b None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:41.362916 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-82092e42-aaea-4d70-9892-71c6847c689b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:41.463291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 273/543] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:41.874164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0aff7-5bb0-4ff8-9929-02396cba6f52 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:41.930006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:41.980450 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0aff7-5bb0-4ff8-9929-02396cba6f52 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:42.016730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4322e-1b49-41e7-9890-e725888d162f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:42.017385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4322e-1b49-41e7-9890-e725888d162f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:42.017666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4322e-1b49-41e7-9890-e725888d162f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:42.017938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4322e-1b49-41e7-9890-e725888d162f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:42.030914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb4322e-1b49-41e7-9890-e725888d162f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:42.035924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-0cb4322e-1b49-41e7-9890-e725888d162f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:42.035924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:42.035924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:42.035924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:42.035924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:42.035924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:42.035924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:42.035924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:42.035924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:42.035924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:42.035924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:42.035924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:42.035924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:42.035924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:42.035924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:42.035924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:42.036880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:42.036880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:42.036880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:42.036880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:42.036880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:42.036880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:42.036880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:42.036880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:42.036880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:42.036880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:42.036880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:42.036880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:42.036880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:42.036880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:42.036880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:42.036880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:42.038109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:42.038109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:42.038109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:42.038109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:42.038109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:42.038109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:42.038109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:42.038109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:42.038109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:42.038109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:42.038109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:42.038109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:42.038109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 271/544] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:42.044035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82092e42-aaea-4d70-9892-71c6847c689b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:42.052536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:42.142952 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f0861-07d1-4546-8244-ab2305d6490f None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:42.143244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f0861-07d1-4546-8244-ab2305d6490f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:42.143474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f0861-07d1-4546-8244-ab2305d6490f None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:42.144989 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 272/545] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:42.151869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97c8d3-df0c-4dcb-adcf-54e4846c04af None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:42.152147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97c8d3-df0c-4dcb-adcf-54e4846c04af None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:42.152421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97c8d3-df0c-4dcb-adcf-54e4846c04af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:42.178156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8f97c8d3-df0c-4dcb-adcf-54e4846c04af None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:42.180665 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8f97c8d3-df0c-4dcb-adcf-54e4846c04af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:42.198028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82092e42-aaea-4d70-9892-71c6847c689b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:42.263083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7529c5-a2f8-40ab-88fa-82c47f1e424f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:42.263724 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7529c5-a2f8-40ab-88fa-82c47f1e424f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:42.264181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7529c5-a2f8-40ab-88fa-82c47f1e424f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:42.264596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7529c5-a2f8-40ab-88fa-82c47f1e424f None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:42.265413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7529c5-a2f8-40ab-88fa-82c47f1e424f None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:42.265864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7529c5-a2f8-40ab-88fa-82c47f1e424f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:42.267359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7529c5-a2f8-40ab-88fa-82c47f1e424f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:42.282592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 274/546] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:08:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:42.288199 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7529c5-a2f8-40ab-88fa-82c47f1e424f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:42.325009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 273/547] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:42.392172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:42.440202 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7529c5-a2f8-40ab-88fa-82c47f1e424f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:42.486469 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64286b1-f489-47de-84d0-fb39efd388a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:42.487097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64286b1-f489-47de-84d0-fb39efd388a6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:42.487379 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64286b1-f489-47de-84d0-fb39efd388a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:42.487682 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64286b1-f489-47de-84d0-fb39efd388a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:42.501408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64286b1-f489-47de-84d0-fb39efd388a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:42.502855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64286b1-f489-47de-84d0-fb39efd388a6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:42.521518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-f64286b1-f489-47de-84d0-fb39efd388a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:42.521518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:42.521518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:42.521518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:42.521518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:42.521518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:42.521518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:42.521518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:42.521518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:42.521518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:42.521518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:42.521518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:42.521518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:42.521518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:42.521518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:42.521518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:42.522454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:42.523301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:42.523301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:42.523301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:42.523301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 275/548] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:42.528826 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97c8d3-df0c-4dcb-adcf-54e4846c04af None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:42.593710 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:42.600776 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64286b1-f489-47de-84d0-fb39efd388a6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:42.718978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97c8d3-df0c-4dcb-adcf-54e4846c04af None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:42.762281 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64286b1-f489-47de-84d0-fb39efd388a6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:42.770280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5264c5d-d3df-4665-8007-7e9a14ba7f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:42.771225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5264c5d-d3df-4665-8007-7e9a14ba7f2b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:42.772085 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5264c5d-d3df-4665-8007-7e9a14ba7f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:42.772592 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5264c5d-d3df-4665-8007-7e9a14ba7f2b None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:42.774355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5264c5d-d3df-4665-8007-7e9a14ba7f2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:42.775111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5264c5d-d3df-4665-8007-7e9a14ba7f2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:42.776892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5264c5d-d3df-4665-8007-7e9a14ba7f2b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:42.784107 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:42.794402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 274/549] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:42.802238 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5264c5d-d3df-4665-8007-7e9a14ba7f2b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:42.804733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c9ff8-ef77-4137-8129-7002edf360f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:42.805552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c9ff8-ef77-4137-8129-7002edf360f8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:42.806007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c9ff8-ef77-4137-8129-7002edf360f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:42.806428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c9ff8-ef77-4137-8129-7002edf360f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:42.817745 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c9ff8-ef77-4137-8129-7002edf360f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:42.823932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-9f6c9ff8-ef77-4137-8129-7002edf360f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:42.823932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:42.823932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:42.823932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:42.823932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:42.823932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:42.823932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:42.823932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:42.823932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:42.823932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:42.823932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:42.823932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:42.823932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:42.823932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:42.823932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:42.823932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:42.825403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:42.825403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:42.825403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:42.825403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:42.825403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:42.825403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:42.825403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:42.825403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:42.825403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:42.825403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:42.825403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:42.825403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:42.825403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:42.825403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:42.825403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:42.825403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:42.826406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:42.826406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:42.826406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:42.826406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:42.826406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:42.826406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:42.826406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:42.826406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:42.826406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:42.826406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:42.826406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:42.826406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:42.833631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 276/550] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:08:42.835022 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c9ff8-ef77-4137-8129-7002edf360f8 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:42.941059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c9ff8-ef77-4137-8129-7002edf360f8 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:42.979130 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35381025-dbeb-41ed-8e1f-755db8ba83f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:42.980132 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35381025-dbeb-41ed-8e1f-755db8ba83f4 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:42.980650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35381025-dbeb-41ed-8e1f-755db8ba83f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:42.981129 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35381025-dbeb-41ed-8e1f-755db8ba83f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:42.981990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35381025-dbeb-41ed-8e1f-755db8ba83f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:42.982499 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35381025-dbeb-41ed-8e1f-755db8ba83f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:42.984273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35381025-dbeb-41ed-8e1f-755db8ba83f4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:42.991444 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5264c5d-d3df-4665-8007-7e9a14ba7f2b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:42.998366 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 277/551] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:08:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:42.999825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:43.000955 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35381025-dbeb-41ed-8e1f-755db8ba83f4 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:43.048527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:43.068554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae3e90-1a3b-4b8a-a59a-be3ba574d704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:43.069215 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae3e90-1a3b-4b8a-a59a-be3ba574d704 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:43.069494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae3e90-1a3b-4b8a-a59a-be3ba574d704 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:43.069819 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae3e90-1a3b-4b8a-a59a-be3ba574d704 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:43.087332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae3e90-1a3b-4b8a-a59a-be3ba574d704 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:43.088870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae3e90-1a3b-4b8a-a59a-be3ba574d704 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:43.111705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-ceae3e90-1a3b-4b8a-a59a-be3ba574d704 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 12:08:43.111705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:43.111705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:43.111705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:43.111705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:43.111705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:43.111705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:43.111705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:43.111705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:43.111705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:43.111705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:43.111705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:43.111705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:08:43.111705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:08:43.111705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:08:43.111705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:43.112884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:43.114432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:43.114432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:43.114432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:08:43.114432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:08:43.114432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:08:43.114432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:08:43.114432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:08:43.114432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:08:43.114432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 12:08:43.114432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:43.114432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 275/552] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:08:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:43.118863 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae3e90-1a3b-4b8a-a59a-be3ba574d704 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:43.124093 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35381025-dbeb-41ed-8e1f-755db8ba83f4 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:43.184274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d822c2-221c-4cea-968c-9ff62ef4efeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:43.184907 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d822c2-221c-4cea-968c-9ff62ef4efeb None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:43.185194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d822c2-221c-4cea-968c-9ff62ef4efeb None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:43.185470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d822c2-221c-4cea-968c-9ff62ef4efeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:43.202402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d822c2-221c-4cea-968c-9ff62ef4efeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:43.203920 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d822c2-221c-4cea-968c-9ff62ef4efeb None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:43.219912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-b8d822c2-221c-4cea-968c-9ff62ef4efeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:43.219912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:43.219912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:43.219912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:43.219912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:43.219912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:43.219912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:43.219912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:43.219912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:43.219912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:43.219912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:43.219912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:43.219912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:43.219912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:43.219912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:43.219912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:43.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:43.225800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:43.225800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:43.225800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:43.225800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 278/553] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:43.225800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf33a96-bf8d-43af-92d9-54c282f8aeeb None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:43.225800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf33a96-bf8d-43af-92d9-54c282f8aeeb None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:43.225800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf33a96-bf8d-43af-92d9-54c282f8aeeb None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:43.225800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 279/554] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:43.230373 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994090ef-2875-4907-9193-e2c01fdcda91 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:43.230656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994090ef-2875-4907-9193-e2c01fdcda91 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:43.230930 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994090ef-2875-4907-9193-e2c01fdcda91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:43.256756 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-994090ef-2875-4907-9193-e2c01fdcda91 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:43.258010 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-994090ef-2875-4907-9193-e2c01fdcda91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:43.288664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae3e90-1a3b-4b8a-a59a-be3ba574d704 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:43.333370 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab33dec-eac6-4462-be14-5ca2bf744e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:43.333994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab33dec-eac6-4462-be14-5ca2bf744e50 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:43.334278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab33dec-eac6-4462-be14-5ca2bf744e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:43.334556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab33dec-eac6-4462-be14-5ca2bf744e50 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:43.335305 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab33dec-eac6-4462-be14-5ca2bf744e50 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:43.335636 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab33dec-eac6-4462-be14-5ca2bf744e50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:43.336756 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 280/555] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:43.337414 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994090ef-2875-4907-9193-e2c01fdcda91 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:43.340175 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab33dec-eac6-4462-be14-5ca2bf744e50 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:43.367213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:43.369496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 276/556] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Oct 6 12:08:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:43.371944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f279c4-9693-4782-9f4f-e93b53b3b6ce None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:43.372462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f279c4-9693-4782-9f4f-e93b53b3b6ce None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:43.373102 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f279c4-9693-4782-9f4f-e93b53b3b6ce None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:43.375524 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 277/557] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:43.376596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f279c4-9693-4782-9f4f-e93b53b3b6ce None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:43.403201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:43.465793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994090ef-2875-4907-9193-e2c01fdcda91 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:43.500443 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f279c4-9693-4782-9f4f-e93b53b3b6ce None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:43.504084 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324c256-4e2b-4550-a62a-fecb54b8113f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:43.504084 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324c256-4e2b-4550-a62a-fecb54b8113f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:43.504418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324c256-4e2b-4550-a62a-fecb54b8113f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:43.504602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324c256-4e2b-4550-a62a-fecb54b8113f None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:43.505238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324c256-4e2b-4550-a62a-fecb54b8113f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:43.505537 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324c256-4e2b-4550-a62a-fecb54b8113f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:43.506895 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324c256-4e2b-4550-a62a-fecb54b8113f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:43.524188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 281/558] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:43.524188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bea81-5506-45eb-9edb-34bc0b4400c5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:43.524188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bea81-5506-45eb-9edb-34bc0b4400c5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:43.524188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bea81-5506-45eb-9edb-34bc0b4400c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:43.542522 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3c6bea81-5506-45eb-9edb-34bc0b4400c5 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:43.543983 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3c6bea81-5506-45eb-9edb-34bc0b4400c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:43.549015 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda6e4e-25ca-454d-a979-1fe8580957fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:43.550157 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda6e4e-25ca-454d-a979-1fe8580957fe None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:43.550975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda6e4e-25ca-454d-a979-1fe8580957fe None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:43.551475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda6e4e-25ca-454d-a979-1fe8580957fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:43.622068 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 282/559] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:43.623289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bea81-5506-45eb-9edb-34bc0b4400c5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:43.638579 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda6e4e-25ca-454d-a979-1fe8580957fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:43.640275 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda6e4e-25ca-454d-a979-1fe8580957fe None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:43.644032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:43.663415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:43.754544 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 278/560] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:43.755673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda6e4e-25ca-454d-a979-1fe8580957fe None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:43.766168 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bea81-5506-45eb-9edb-34bc0b4400c5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:43.814364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080f5ba-6d84-4685-ad14-f0512c597f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:43.815390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080f5ba-6d84-4685-ad14-f0512c597f27 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:43.815810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080f5ba-6d84-4685-ad14-f0512c597f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:43.816176 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080f5ba-6d84-4685-ad14-f0512c597f27 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:43.827218 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080f5ba-6d84-4685-ad14-f0512c597f27 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:43.829619 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080f5ba-6d84-4685-ad14-f0512c597f27 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:43.846603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-d080f5ba-6d84-4685-ad14-f0512c597f27 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:08:43.846603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:43.846603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:43.846603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:43.846603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:43.846603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:43.846603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:43.846603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:43.846603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:43.846603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:43.846603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:43.846603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:43.846603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:08:43.846603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:08:43.846603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:08:43.846603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:43.847717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:43.848721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:43.848721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:43.848721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:08:43.848721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:08:43.848721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:08:43.848721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:08:43.848721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:08:43.848721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:08:43.848721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:08:43.848721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:43.851613 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 283/561] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Oct 6 12:08:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:43.852714 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080f5ba-6d84-4685-ad14-f0512c597f27 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:43.871970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda6e4e-25ca-454d-a979-1fe8580957fe None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:43.893717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:43.913585 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82d29f-878c-4969-9d23-ffc18f34fb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:43.914275 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82d29f-878c-4969-9d23-ffc18f34fb51 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:43.914561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82d29f-878c-4969-9d23-ffc18f34fb51 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:43.914831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82d29f-878c-4969-9d23-ffc18f34fb51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:43.925320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82d29f-878c-4969-9d23-ffc18f34fb51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:43.930789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-8e82d29f-878c-4969-9d23-ffc18f34fb51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:43.930789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:43.930789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:43.930789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:43.930789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:43.930789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:43.930789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:43.930789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:43.930789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:43.930789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:43.930789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:43.930789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:43.930789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:43.930789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:43.930789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:43.930789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:43.931921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:43.931921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:43.931921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:43.931921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:43.931921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:43.931921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:43.931921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:43.931921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:43.931921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:43.931921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:43.931921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:43.931921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:43.931921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:43.931921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:43.931921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:43.931921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:43.932816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:43.932816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:43.932816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:43.932816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:43.932816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:43.932816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:43.932816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:43.932816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:43.932816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:43.932816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:43.932816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:43.932816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:43.932816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 279/562] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:43.933540 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82d29f-878c-4969-9d23-ffc18f34fb51 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:43.952801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:43.977930 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080f5ba-6d84-4685-ad14-f0512c597f27 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:44.013394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b253461-5059-49c7-84e5-2afb7883e587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:44.013954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b253461-5059-49c7-84e5-2afb7883e587 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:44.014236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b253461-5059-49c7-84e5-2afb7883e587 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:44.014506 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b253461-5059-49c7-84e5-2afb7883e587 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:44.025508 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b253461-5059-49c7-84e5-2afb7883e587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:44.026960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b253461-5059-49c7-84e5-2afb7883e587 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:44.030868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82d29f-878c-4969-9d23-ffc18f34fb51 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:44.041091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-5b253461-5059-49c7-84e5-2afb7883e587 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:44.041091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:44.041091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:44.041091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:44.041091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:44.041091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:44.041091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:44.041091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:44.041091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:44.041091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:44.041091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:44.041091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:44.041091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:44.041091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:44.041091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:44.041091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:44.042122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:44.043041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:44.043041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:44.043041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:44.043041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 284/563] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:44.043041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b253461-5059-49c7-84e5-2afb7883e587 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:44.067571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82d892-b148-4587-bd44-6d73d920b942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:44.068409 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82d892-b148-4587-bd44-6d73d920b942 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:44.068694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82d892-b148-4587-bd44-6d73d920b942 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:44.068968 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82d892-b148-4587-bd44-6d73d920b942 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:44.069687 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82d892-b148-4587-bd44-6d73d920b942 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:44.069999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82d892-b148-4587-bd44-6d73d920b942 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:44.071456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82d892-b148-4587-bd44-6d73d920b942 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:44.088146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 280/564] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Oct 6 12:08:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:44.089329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f9f4f-d15b-46be-b856-8a687c0755d3 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:44.089619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f9f4f-d15b-46be-b856-8a687c0755d3 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:44.089898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f9f4f-d15b-46be-b856-8a687c0755d3 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:44.091230 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 281/565] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:44.091803 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f9f4f-d15b-46be-b856-8a687c0755d3 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:44.131686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:44.146992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b253461-5059-49c7-84e5-2afb7883e587 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:44.172195 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:44.181602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec63402-cda7-4c48-a7a3-e7d493cba049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:44.182364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec63402-cda7-4c48-a7a3-e7d493cba049 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:44.182957 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec63402-cda7-4c48-a7a3-e7d493cba049 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:44.183230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec63402-cda7-4c48-a7a3-e7d493cba049 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:44.191726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f9f4f-d15b-46be-b856-8a687c0755d3 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:44.193889 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec63402-cda7-4c48-a7a3-e7d493cba049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:44.199658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-cec63402-cda7-4c48-a7a3-e7d493cba049 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:44.199658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:44.199658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:44.199658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:44.199658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:44.199658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:44.199658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:44.199658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:44.199658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:44.199658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:44.199658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:44.199658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:44.199658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:44.199658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:44.199658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:44.199658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:44.200770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:44.200770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:44.200770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:44.200770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:44.200770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:44.200770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:44.200770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:44.200770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:44.200770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:44.200770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:44.200770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:44.200770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:44.200770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:44.200770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:44.200770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:44.200770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:44.201648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:44.201648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:44.201648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:44.201648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:44.201648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:44.201648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:44.201648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:44.201648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:44.201648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:44.201648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:44.201648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:44.201648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:44.201648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 285/566] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:44.201648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec63402-cda7-4c48-a7a3-e7d493cba049 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:44.227668 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c149f32-e84a-41d2-8608-7dd88acc22ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:44.228275 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c149f32-e84a-41d2-8608-7dd88acc22ed None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:44.228556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c149f32-e84a-41d2-8608-7dd88acc22ed None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:44.228833 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c149f32-e84a-41d2-8608-7dd88acc22ed None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:44.229493 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c149f32-e84a-41d2-8608-7dd88acc22ed None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:44.229797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c149f32-e84a-41d2-8608-7dd88acc22ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:44.231155 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c149f32-e84a-41d2-8608-7dd88acc22ed None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:44.238966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 282/567] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:08:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:44.239624 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c149f32-e84a-41d2-8608-7dd88acc22ed None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:44.308886 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec63402-cda7-4c48-a7a3-e7d493cba049 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:44.333282 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:44.336405 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c149f32-e84a-41d2-8608-7dd88acc22ed None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:44.342172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca85a0f-8c28-4712-8e42-3c7274776198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:44.342778 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca85a0f-8c28-4712-8e42-3c7274776198 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:44.343063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca85a0f-8c28-4712-8e42-3c7274776198 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:44.343337 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca85a0f-8c28-4712-8e42-3c7274776198 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:44.344008 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca85a0f-8c28-4712-8e42-3c7274776198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:44.344319 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca85a0f-8c28-4712-8e42-3c7274776198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:44.345690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca85a0f-8c28-4712-8e42-3c7274776198 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:44.357883 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 286/568] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:44.359039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d06d1b-8849-430f-8b8b-92454eb6ccac None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:44.359325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d06d1b-8849-430f-8b8b-92454eb6ccac None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:44.359633 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d06d1b-8849-430f-8b8b-92454eb6ccac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:44.373743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8d37f-e9dc-47d5-81f5-968d9a36eb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:44.374384 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8d37f-e9dc-47d5-81f5-968d9a36eb8c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:44.374652 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8d37f-e9dc-47d5-81f5-968d9a36eb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:44.374924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8d37f-e9dc-47d5-81f5-968d9a36eb8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:44.376898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e6d06d1b-8849-430f-8b8b-92454eb6ccac None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:44.378081 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e6d06d1b-8849-430f-8b8b-92454eb6ccac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:44.385327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8d37f-e9dc-47d5-81f5-968d9a36eb8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:44.386774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8d37f-e9dc-47d5-81f5-968d9a36eb8c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:44.402294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-7fa8d37f-e9dc-47d5-81f5-968d9a36eb8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:44.402294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:44.402294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:44.402294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:44.402294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:44.402294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:44.402294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:44.402294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:44.402294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:44.402294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:44.402294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:44.402294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:44.402294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:44.402294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:44.402294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:44.402294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:44.403272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:44.404216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:44.404216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:44.404216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:44.404216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 283/569] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:44.404216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8d37f-e9dc-47d5-81f5-968d9a36eb8c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:44.426954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:44.452156 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 287/570] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:44.452976 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d06d1b-8849-430f-8b8b-92454eb6ccac None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:44.506198 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8d37f-e9dc-47d5-81f5-968d9a36eb8c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:44.542222 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f38ec-dbeb-4d17-b55c-a05a6548a0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:44.542825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f38ec-dbeb-4d17-b55c-a05a6548a0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:44.543110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f38ec-dbeb-4d17-b55c-a05a6548a0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:44.543382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f38ec-dbeb-4d17-b55c-a05a6548a0e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:44.544072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f38ec-dbeb-4d17-b55c-a05a6548a0e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:44.544371 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f38ec-dbeb-4d17-b55c-a05a6548a0e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:44.545730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f38ec-dbeb-4d17-b55c-a05a6548a0e1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:44.558384 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 284/571] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:08:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:44.558384 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f38ec-dbeb-4d17-b55c-a05a6548a0e1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:44.569843 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d06d1b-8849-430f-8b8b-92454eb6ccac None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:44.573318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:44.609293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec03047-cecb-4c01-af72-8a9fa355c5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:44.610174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec03047-cecb-4c01-af72-8a9fa355c5bb None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:44.610846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec03047-cecb-4c01-af72-8a9fa355c5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:44.611126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec03047-cecb-4c01-af72-8a9fa355c5bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:44.624032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec03047-cecb-4c01-af72-8a9fa355c5bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:44.625472 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec03047-cecb-4c01-af72-8a9fa355c5bb None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:44.643138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-dec03047-cecb-4c01-af72-8a9fa355c5bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:44.643138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:44.643138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:44.643138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:44.643138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:44.643138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:44.643138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:44.643138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:44.643138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:44.643138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:44.643138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:44.643138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:44.643138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:44.643138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:44.643138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:44.643138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:44.644279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:44.645198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:44.645198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:44.645198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:44.647806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 288/572] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:44.648937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec03047-cecb-4c01-af72-8a9fa355c5bb None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:44.662149 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f38ec-dbeb-4d17-b55c-a05a6548a0e1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:44.678245 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:44.699253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdaea2-03c5-4f6b-968f-ed7ed6876766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:44.700225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdaea2-03c5-4f6b-968f-ed7ed6876766 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:44.700741 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdaea2-03c5-4f6b-968f-ed7ed6876766 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:44.701232 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdaea2-03c5-4f6b-968f-ed7ed6876766 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:44.708102 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdaea2-03c5-4f6b-968f-ed7ed6876766 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:44.709646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdaea2-03c5-4f6b-968f-ed7ed6876766 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:44.720965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-2cfdaea2-03c5-4f6b-968f-ed7ed6876766 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:44.720965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:44.720965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:44.720965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:44.720965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:44.720965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:44.720965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:44.720965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:44.720965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:44.720965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:44.720965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:44.720965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:44.720965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:08:44.720965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:08:44.720965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:44.721913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:44.723593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:08:44.723593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:08:44.723593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:08:44.723593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:08:44.723593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:08:44.723593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:08:44.723593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:44.723593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:44.723593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 285/573] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:44.723593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdaea2-03c5-4f6b-968f-ed7ed6876766 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:44.759190 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec03047-cecb-4c01-af72-8a9fa355c5bb None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:44.792361 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7ecac-a1a8-44b9-99d9-8107cd690202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:44.792961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7ecac-a1a8-44b9-99d9-8107cd690202 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:44.793243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7ecac-a1a8-44b9-99d9-8107cd690202 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:44.793517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7ecac-a1a8-44b9-99d9-8107cd690202 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:44.794152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7ecac-a1a8-44b9-99d9-8107cd690202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:44.794456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7ecac-a1a8-44b9-99d9-8107cd690202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:44.795137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:44.795851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7ecac-a1a8-44b9-99d9-8107cd690202 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:44.808090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 289/574] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:44.808878 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7ecac-a1a8-44b9-99d9-8107cd690202 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:44.824775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdaea2-03c5-4f6b-968f-ed7ed6876766 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:44.857058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf754c-0ed4-463a-8cff-4cc22a19227e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:44.857746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf754c-0ed4-463a-8cff-4cc22a19227e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:44.858033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf754c-0ed4-463a-8cff-4cc22a19227e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:44.858308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf754c-0ed4-463a-8cff-4cc22a19227e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:44.869526 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf754c-0ed4-463a-8cff-4cc22a19227e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:44.870968 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf754c-0ed4-463a-8cff-4cc22a19227e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:44.884856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-0faf754c-0ed4-463a-8cff-4cc22a19227e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:44.884856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:44.884856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:44.884856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:44.884856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:44.884856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:44.884856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:44.884856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:44.884856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:44.884856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:44.884856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:44.884856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:44.884856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:44.884856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:44.884856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:44.884856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:44.886039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:44.887259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:44.887259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:44.887259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:44.887259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 286/575] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:44.887259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf754c-0ed4-463a-8cff-4cc22a19227e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:44.895957 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:44.908752 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7ecac-a1a8-44b9-99d9-8107cd690202 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:44.943358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbfed3-a8cd-4b9f-9852-8378fab8621e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:44.944008 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbfed3-a8cd-4b9f-9852-8378fab8621e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:44.944292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbfed3-a8cd-4b9f-9852-8378fab8621e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:44.944573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbfed3-a8cd-4b9f-9852-8378fab8621e None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:44.945205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbfed3-a8cd-4b9f-9852-8378fab8621e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:44.946041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbfed3-a8cd-4b9f-9852-8378fab8621e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:44.947395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbfed3-a8cd-4b9f-9852-8378fab8621e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:44.958806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 290/576] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:44.959702 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbfed3-a8cd-4b9f-9852-8378fab8621e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:44.987595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf754c-0ed4-463a-8cff-4cc22a19227e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:45.011740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:45.020558 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b1f9f-9ddb-4408-aca7-d018e30ccb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:45.021176 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b1f9f-9ddb-4408-aca7-d018e30ccb5b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:45.021452 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b1f9f-9ddb-4408-aca7-d018e30ccb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:45.021720 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b1f9f-9ddb-4408-aca7-d018e30ccb5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:45.031865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b1f9f-9ddb-4408-aca7-d018e30ccb5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:45.037008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-2a8b1f9f-9ddb-4408-aca7-d018e30ccb5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:45.037008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:45.037008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:45.037008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:45.037008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:45.037008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:45.037008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:45.037008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:45.037008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:45.037008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:45.037008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:45.037008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:45.037008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:45.037008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:45.037008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:45.037008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:45.038129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:45.038129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:45.038129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:45.038129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:45.038129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:45.038129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:45.038129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:45.038129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:45.038129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:45.038129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:45.038129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:45.038129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:45.038129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:45.038129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:45.038129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:45.038129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:45.039911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:45.039911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:45.039911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:45.039911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:45.039911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:45.039911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:45.039911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:45.039911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:45.039911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:45.039911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:45.039911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:45.039911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:45.039911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 287/577] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:45.039911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b1f9f-9ddb-4408-aca7-d018e30ccb5b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:45.062809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbfed3-a8cd-4b9f-9852-8378fab8621e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:45.095834 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee82db1-a332-4979-921a-339ba323901f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:45.096366 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee82db1-a332-4979-921a-339ba323901f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:45.096641 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee82db1-a332-4979-921a-339ba323901f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:45.096913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee82db1-a332-4979-921a-339ba323901f None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:45.097594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee82db1-a332-4979-921a-339ba323901f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:45.097896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee82db1-a332-4979-921a-339ba323901f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:45.099299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee82db1-a332-4979-921a-339ba323901f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:45.111270 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 291/578] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:08:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:45.112074 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee82db1-a332-4979-921a-339ba323901f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:45.126542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:45.137833 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b1f9f-9ddb-4408-aca7-d018e30ccb5b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:45.170733 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941afd41-0d04-4ed0-b838-72dbf90c148b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:45.171355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941afd41-0d04-4ed0-b838-72dbf90c148b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:45.171693 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941afd41-0d04-4ed0-b838-72dbf90c148b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:45.171960 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941afd41-0d04-4ed0-b838-72dbf90c148b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:45.178783 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941afd41-0d04-4ed0-b838-72dbf90c148b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:45.180314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941afd41-0d04-4ed0-b838-72dbf90c148b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:45.191455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-941afd41-0d04-4ed0-b838-72dbf90c148b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:45.191455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:45.191455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:45.191455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:45.191455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:45.191455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:45.191455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:45.191455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:45.191455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:45.191455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:45.191455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:45.191455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:45.191455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:08:45.191455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:08:45.191455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:45.192573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:45.198271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:08:45.198271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:08:45.198271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:08:45.198271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:08:45.198271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:08:45.198271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:08:45.198271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:45.198271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:45.198271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 288/579] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:45.198271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941afd41-0d04-4ed0-b838-72dbf90c148b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:45.231159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee82db1-a332-4979-921a-339ba323901f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:45.240546 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:45.269857 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a5ddf-c729-4f26-a985-4e1cac6d0836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:45.270467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a5ddf-c729-4f26-a985-4e1cac6d0836 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:45.270753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a5ddf-c729-4f26-a985-4e1cac6d0836 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:45.271023 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a5ddf-c729-4f26-a985-4e1cac6d0836 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:45.271708 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a5ddf-c729-4f26-a985-4e1cac6d0836 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:45.272016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a5ddf-c729-4f26-a985-4e1cac6d0836 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:45.273398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a5ddf-c729-4f26-a985-4e1cac6d0836 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:45.284879 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 292/580] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:45.285662 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a5ddf-c729-4f26-a985-4e1cac6d0836 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:45.296299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941afd41-0d04-4ed0-b838-72dbf90c148b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:45.329304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fa851-ffd6-4c05-bc4e-8d2e4aae179f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:45.329926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fa851-ffd6-4c05-bc4e-8d2e4aae179f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:45.330201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fa851-ffd6-4c05-bc4e-8d2e4aae179f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:45.330470 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fa851-ffd6-4c05-bc4e-8d2e4aae179f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:45.338124 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fa851-ffd6-4c05-bc4e-8d2e4aae179f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:45.339571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fa851-ffd6-4c05-bc4e-8d2e4aae179f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:45.349205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:45.350571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-e35fa851-ffd6-4c05-bc4e-8d2e4aae179f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 12:08:45.350571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:45.350571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:45.350571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:45.350571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:45.350571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:45.350571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:45.350571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:45.350571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:45.350571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:45.350571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:45.350571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:45.350571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:08:45.350571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:08:45.350571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:08:45.350571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:45.352122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:45.353098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:45.353098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:45.353098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:08:45.353098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:08:45.353098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:08:45.353098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:08:45.353098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:08:45.353098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:08:45.353098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 12:08:45.353098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:45.356564 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 289/581] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:45.357288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fa851-ffd6-4c05-bc4e-8d2e4aae179f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:45.382956 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a5ddf-c729-4f26-a985-4e1cac6d0836 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:45.417529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae37d6-d912-421b-86d4-511990c3c9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:45.418135 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae37d6-d912-421b-86d4-511990c3c9ea None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:45.418419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae37d6-d912-421b-86d4-511990c3c9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:45.418696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae37d6-d912-421b-86d4-511990c3c9ea None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:45.419352 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae37d6-d912-421b-86d4-511990c3c9ea None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:45.419697 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae37d6-d912-421b-86d4-511990c3c9ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:45.421076 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae37d6-d912-421b-86d4-511990c3c9ea None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:45.428756 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 293/582] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:08:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:45.429539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae37d6-d912-421b-86d4-511990c3c9ea None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:45.458698 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fa851-ffd6-4c05-bc4e-8d2e4aae179f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:45.466169 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:45.492949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28138c5-33c5-4420-b427-b4e376bd7230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:45.493543 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28138c5-33c5-4420-b427-b4e376bd7230 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:45.493820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28138c5-33c5-4420-b427-b4e376bd7230 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:45.494090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28138c5-33c5-4420-b427-b4e376bd7230 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:45.494804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28138c5-33c5-4420-b427-b4e376bd7230 None admin] RBAC: Adding query filter params (role.id=910821ade2904d23b75fa69e040fb82d, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=3deb4b248f884c19bf1e444f2fe0923a) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:45.495109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28138c5-33c5-4420-b427-b4e376bd7230 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:45.496555 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28138c5-33c5-4420-b427-b4e376bd7230 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:45.501663 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 290/583] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:08:45 2022] GET /identity/v3/role_assignments?user.id=3deb4b248f884c19bf1e444f2fe0923a&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=910821ade2904d23b75fa69e040fb82d => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:45.502422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28138c5-33c5-4420-b427-b4e376bd7230 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:45.535300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae37d6-d912-421b-86d4-511990c3c9ea None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:45.568347 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:45.576063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d57f4d-6cfa-4702-80f9-e79134378d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:45.576841 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d57f4d-6cfa-4702-80f9-e79134378d50 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:45.577011 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d57f4d-6cfa-4702-80f9-e79134378d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:45.577276 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d57f4d-6cfa-4702-80f9-e79134378d50 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:45.577990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d57f4d-6cfa-4702-80f9-e79134378d50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:45.578295 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d57f4d-6cfa-4702-80f9-e79134378d50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:45.580171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d57f4d-6cfa-4702-80f9-e79134378d50 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:45.599558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 294/584] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:08:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:45.599558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d57f4d-6cfa-4702-80f9-e79134378d50 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:45.625584 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28138c5-33c5-4420-b427-b4e376bd7230 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:45.663197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae557c-07a3-4032-85a0-0527cb3f5d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:45.664737 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae557c-07a3-4032-85a0-0527cb3f5d0b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:45.665016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae557c-07a3-4032-85a0-0527cb3f5d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:45.665288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae557c-07a3-4032-85a0-0527cb3f5d0b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:45.675840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae557c-07a3-4032-85a0-0527cb3f5d0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:45.678407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae557c-07a3-4032-85a0-0527cb3f5d0b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:45.690666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-bbae557c-07a3-4032-85a0-0527cb3f5d0b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 06 12:08:45.690666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:45.690666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:45.690666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:45.690666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:45.690666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:45.690666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:45.690666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:45.690666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:45.690666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:45.690666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:45.690666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:45.690666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:08:45.690666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:08:45.690666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:08:45.690666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:45.691726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:45.692732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:45.692732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:45.692732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:08:45.692732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:08:45.692732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:08:45.692732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:08:45.692732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:08:45.692732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:08:45.692732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 06 12:08:45.692732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:45.692732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 291/585] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:45.692732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae557c-07a3-4032-85a0-0527cb3f5d0b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:45.718968 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d57f4d-6cfa-4702-80f9-e79134378d50 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:45.723993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:45.755921 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaac936-493c-4b61-b190-bd95c97f7f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:45.756534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaac936-493c-4b61-b190-bd95c97f7f7f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:45.756924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaac936-493c-4b61-b190-bd95c97f7f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:45.757289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaac936-493c-4b61-b190-bd95c97f7f7f None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:45.758193 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaac936-493c-4b61-b190-bd95c97f7f7f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:45.758506 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaac936-493c-4b61-b190-bd95c97f7f7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:45.759968 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaac936-493c-4b61-b190-bd95c97f7f7f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:45.783287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 295/586] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Oct 6 12:08:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:45.784452 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaac936-493c-4b61-b190-bd95c97f7f7f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:45.797328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae557c-07a3-4032-85a0-0527cb3f5d0b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:45.831965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c999d3-b993-4296-ad2d-caa998a07b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:45.832587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c999d3-b993-4296-ad2d-caa998a07b65 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:45.832887 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c999d3-b993-4296-ad2d-caa998a07b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:45.833131 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c999d3-b993-4296-ad2d-caa998a07b65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:45.837244 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:45.844714 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c999d3-b993-4296-ad2d-caa998a07b65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:45.846145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c999d3-b993-4296-ad2d-caa998a07b65 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:45.859964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-b2c999d3-b993-4296-ad2d-caa998a07b65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:45.859964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:45.859964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:45.859964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:45.859964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:45.859964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:45.859964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:45.859964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:45.859964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:45.859964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:45.859964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:45.859964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:45.859964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:45.859964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:45.859964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:45.859964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:45.860929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:45.861835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:45.861835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:45.861835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:45.861835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 292/587] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:45.861835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c999d3-b993-4296-ad2d-caa998a07b65 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:45.892615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaac936-493c-4b61-b190-bd95c97f7f7f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:45.930487 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c2e2f-1c34-4ad1-98b5-ddba920e40a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:45.931145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c2e2f-1c34-4ad1-98b5-ddba920e40a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:45.931434 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c2e2f-1c34-4ad1-98b5-ddba920e40a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:45.931734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c2e2f-1c34-4ad1-98b5-ddba920e40a3 None admin] PATH_INFO: `/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/5a46e63d34564d34b6b5752706375798/roles/910821ade2904d23b75fa69e040fb82d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:45.947369 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c2e2f-1c34-4ad1-98b5-ddba920e40a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=314d3d13ba2e4a0f93f5d3f34c50a96f, user_id=5a46e63d34564d34b6b5752706375798, role_id=910821ade2904d23b75fa69e040fb82d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:45.949165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c2e2f-1c34-4ad1-98b5-ddba920e40a3 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:45.951746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:45.964199 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 296/588] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:08:45 2022] PUT /identity/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/5a46e63d34564d34b6b5752706375798/roles/910821ade2904d23b75fa69e040fb82d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:08:45.964795 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c2e2f-1c34-4ad1-98b5-ddba920e40a3 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:45.982523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c999d3-b993-4296-ad2d-caa998a07b65 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:46.027170 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82241f79-eef0-49c0-8f05-793270c74333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:46.027829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82241f79-eef0-49c0-8f05-793270c74333 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:46.028108 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82241f79-eef0-49c0-8f05-793270c74333 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:46.028377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82241f79-eef0-49c0-8f05-793270c74333 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:46.029126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82241f79-eef0-49c0-8f05-793270c74333 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:46.029435 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82241f79-eef0-49c0-8f05-793270c74333 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:46.031221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82241f79-eef0-49c0-8f05-793270c74333 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:46.056007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 293/589] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Oct 6 12:08:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:46.056993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82241f79-eef0-49c0-8f05-793270c74333 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:46.115171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:46.120019 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c2e2f-1c34-4ad1-98b5-ddba920e40a3 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:46.161942 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7ef74-e7db-4af0-b761-22290a5bfc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:46.162581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7ef74-e7db-4af0-b761-22290a5bfc8a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:46.162863 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7ef74-e7db-4af0-b761-22290a5bfc8a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:46.163134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7ef74-e7db-4af0-b761-22290a5bfc8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:46.176755 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7ef74-e7db-4af0-b761-22290a5bfc8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:46.178501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7ef74-e7db-4af0-b761-22290a5bfc8a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:46.190351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82241f79-eef0-49c0-8f05-793270c74333 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:46.198394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-19b7ef74-e7db-4af0-b761-22290a5bfc8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:46.198394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:46.198394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:46.198394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:46.198394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:46.198394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:46.198394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:46.198394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:46.198394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:46.198394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:46.198394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:46.198394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:46.198394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:46.198394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:46.198394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:46.198394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:46.199377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:46.200293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:46.200293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:46.200293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:46.200293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 297/590] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:46.201361 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7ef74-e7db-4af0-b761-22290a5bfc8a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:46.229946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1753d-6c09-43a0-a7b2-35c993eb43bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:46.230570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1753d-6c09-43a0-a7b2-35c993eb43bc None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:46.231324 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1753d-6c09-43a0-a7b2-35c993eb43bc None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:46.231636 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1753d-6c09-43a0-a7b2-35c993eb43bc None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:46.232570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1753d-6c09-43a0-a7b2-35c993eb43bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:46.232881 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1753d-6c09-43a0-a7b2-35c993eb43bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:46.234763 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1753d-6c09-43a0-a7b2-35c993eb43bc None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:46.251465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:46.257682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 294/591] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:46.262559 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1753d-6c09-43a0-a7b2-35c993eb43bc None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:46.353265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7ef74-e7db-4af0-b761-22290a5bfc8a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:46.370138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1753d-6c09-43a0-a7b2-35c993eb43bc None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:46.390155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd00291-be9f-4c5b-890d-09886535477d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:46.391171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd00291-be9f-4c5b-890d-09886535477d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:46.391705 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd00291-be9f-4c5b-890d-09886535477d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:46.392189 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd00291-be9f-4c5b-890d-09886535477d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:46.405341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd00291-be9f-4c5b-890d-09886535477d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:46.407558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd00291-be9f-4c5b-890d-09886535477d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:46.407808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb624e-d1f6-4f4f-bb0c-65e5b0ef0521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:46.408589 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb624e-d1f6-4f4f-bb0c-65e5b0ef0521 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:46.408916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb624e-d1f6-4f4f-bb0c-65e5b0ef0521 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:46.409253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb624e-d1f6-4f4f-bb0c-65e5b0ef0521 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:46.409998 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb624e-d1f6-4f4f-bb0c-65e5b0ef0521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:46.410363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb624e-d1f6-4f4f-bb0c-65e5b0ef0521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:46.412152 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb624e-d1f6-4f4f-bb0c-65e5b0ef0521 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:46.425205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-ffd00291-be9f-4c5b-890d-09886535477d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:46.425205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:46.425205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:46.425205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:46.425205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:46.425205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:46.425205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:46.425205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:46.425205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:46.425205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:46.425205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:46.425205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:46.425205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:46.425205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:46.425205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:46.425205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:46.426551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:46.427465 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:46.427465 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:46.427465 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:46.427699 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 295/592] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:46.427699 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb624e-d1f6-4f4f-bb0c-65e5b0ef0521 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:46.431589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 298/593] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:46.432641 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd00291-be9f-4c5b-890d-09886535477d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:46.435710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:46.505520 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:46.558784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb624e-d1f6-4f4f-bb0c-65e5b0ef0521 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:46.565108 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd00291-be9f-4c5b-890d-09886535477d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:46.604282 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82143f3a-0bee-4947-9e90-8927d150770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:46.605385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82143f3a-0bee-4947-9e90-8927d150770c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:46.605945 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82143f3a-0bee-4947-9e90-8927d150770c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:46.606320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ced80f-d3d9-4569-b2b2-bdb4852ca0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:46.606948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82143f3a-0bee-4947-9e90-8927d150770c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:46.607053 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ced80f-d3d9-4569-b2b2-bdb4852ca0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:46.607362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ced80f-d3d9-4569-b2b2-bdb4852ca0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:46.607729 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ced80f-d3d9-4569-b2b2-bdb4852ca0e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:46.615301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ced80f-d3d9-4569-b2b2-bdb4852ca0e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:46.615732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82143f3a-0bee-4947-9e90-8927d150770c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:46.616921 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ced80f-d3d9-4569-b2b2-bdb4852ca0e2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:46.617785 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82143f3a-0bee-4947-9e90-8927d150770c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:46.629694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-44ced80f-d3d9-4569-b2b2-bdb4852ca0e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:46.629694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:46.629694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:46.629694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:46.629694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:46.629694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:46.629694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:46.629694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:46.629694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:46.629694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:46.629694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:46.629694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:46.629694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:08:46.629694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:08:46.629694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:46.630623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:46.632006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:08:46.632006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:08:46.632006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:08:46.632006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:08:46.632006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:08:46.632006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:08:46.632006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:46.632006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:46.632006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 299/594] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:46.632006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ced80f-d3d9-4569-b2b2-bdb4852ca0e2 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:46.634308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-82143f3a-0bee-4947-9e90-8927d150770c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 12:08:46.634308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:46.634308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:46.634308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:46.634308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:46.634308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:46.634308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:46.634308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:46.634308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:46.634308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:46.634308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:46.634308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:46.634308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:08:46.634308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:08:46.634308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:08:46.634308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:46.636012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:46.637010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:46.637010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:46.637010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:08:46.637010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:08:46.637010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:08:46.637010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:08:46.637010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:08:46.637010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:08:46.637010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 12:08:46.637010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:46.639824 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 296/595] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:46.640928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82143f3a-0bee-4947-9e90-8927d150770c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:46.711215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:46.761434 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ced80f-d3d9-4569-b2b2-bdb4852ca0e2 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:46.762678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82143f3a-0bee-4947-9e90-8927d150770c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:46.783701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:46.800515 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fee0c8-e44f-4b8e-baa5-a85a5d336586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:46.801265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fee0c8-e44f-4b8e-baa5-a85a5d336586 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:46.801637 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fee0c8-e44f-4b8e-baa5-a85a5d336586 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:46.802016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fee0c8-e44f-4b8e-baa5-a85a5d336586 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:46.802794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fee0c8-e44f-4b8e-baa5-a85a5d336586 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:46.803178 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fee0c8-e44f-4b8e-baa5-a85a5d336586 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:46.805307 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fee0c8-e44f-4b8e-baa5-a85a5d336586 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:46.807409 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35cf2a-d2ad-4901-b19b-3c257413116c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:46.808732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35cf2a-d2ad-4901-b19b-3c257413116c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:46.809167 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35cf2a-d2ad-4901-b19b-3c257413116c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:46.809609 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35cf2a-d2ad-4901-b19b-3c257413116c None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:46.810402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed35cf2a-d2ad-4901-b19b-3c257413116c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:46.810855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed35cf2a-d2ad-4901-b19b-3c257413116c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:46.812381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed35cf2a-d2ad-4901-b19b-3c257413116c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:46.818889 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 297/596] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:08:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:46.819938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fee0c8-e44f-4b8e-baa5-a85a5d336586 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:46.829996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 300/597] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:46.831057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35cf2a-d2ad-4901-b19b-3c257413116c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:46.949049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35cf2a-d2ad-4901-b19b-3c257413116c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:46.953348 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fee0c8-e44f-4b8e-baa5-a85a5d336586 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:46.980453 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:46.989700 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b09067-6d39-4264-95ba-a0c9625a6aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:46.990427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b09067-6d39-4264-95ba-a0c9625a6aac None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:46.990907 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b09067-6d39-4264-95ba-a0c9625a6aac None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:46.991284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b09067-6d39-4264-95ba-a0c9625a6aac None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:46.992677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b09067-6d39-4264-95ba-a0c9625a6aac None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:46.993077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b09067-6d39-4264-95ba-a0c9625a6aac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:46.994585 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b09067-6d39-4264-95ba-a0c9625a6aac None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:46.997941 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5467f3-f63d-49bb-b349-7528f2a661a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:46.998768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5467f3-f63d-49bb-b349-7528f2a661a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:46.999219 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5467f3-f63d-49bb-b349-7528f2a661a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:46.999666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5467f3-f63d-49bb-b349-7528f2a661a9 None admin] PATH_INFO: `/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/43e873f881e6497ea09f7f10f08e3bd4/roles/910821ade2904d23b75fa69e040fb82d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:47.012740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 298/598] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Oct 6 12:08:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:47.013766 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b09067-6d39-4264-95ba-a0c9625a6aac None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:47.018600 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5467f3-f63d-49bb-b349-7528f2a661a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=314d3d13ba2e4a0f93f5d3f34c50a96f, user_id=43e873f881e6497ea09f7f10f08e3bd4, role_id=910821ade2904d23b75fa69e040fb82d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:47.020267 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5467f3-f63d-49bb-b349-7528f2a661a9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:47.037429 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 301/599] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:08:46 2022] PUT /identity/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/43e873f881e6497ea09f7f10f08e3bd4/roles/910821ade2904d23b75fa69e040fb82d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:08:47.038488 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5467f3-f63d-49bb-b349-7528f2a661a9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:47.062290 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:47.180197 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5467f3-f63d-49bb-b349-7528f2a661a9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:47.230628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5869d77-c0eb-425a-91c0-7c0bba6906a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:47.231654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5869d77-c0eb-425a-91c0-7c0bba6906a7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:47.232127 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5869d77-c0eb-425a-91c0-7c0bba6906a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:47.232572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5869d77-c0eb-425a-91c0-7c0bba6906a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:47.234057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b09067-6d39-4264-95ba-a0c9625a6aac None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:47.254767 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5869d77-c0eb-425a-91c0-7c0bba6906a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:47.257386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5869d77-c0eb-425a-91c0-7c0bba6906a7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:47.274580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-b5869d77-c0eb-425a-91c0-7c0bba6906a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:47.274580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:47.274580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:47.274580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:47.274580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:47.274580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:47.274580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:47.274580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:47.274580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:47.274580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:47.274580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:47.274580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:47.274580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:47.274580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:47.274580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:47.274580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:47.275840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:47.276945 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:47.276945 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:47.276945 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:47.277975 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 302/600] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:47.283760 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5869d77-c0eb-425a-91c0-7c0bba6906a7 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:47.290130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4202f06-1984-4687-be83-99866cf23ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:47.291003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4202f06-1984-4687-be83-99866cf23ebb None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:47.291461 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4202f06-1984-4687-be83-99866cf23ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:47.291983 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4202f06-1984-4687-be83-99866cf23ebb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:47.300967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4202f06-1984-4687-be83-99866cf23ebb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:47.303317 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4202f06-1984-4687-be83-99866cf23ebb None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:47.315386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:47.323486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-f4202f06-1984-4687-be83-99866cf23ebb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:47.323486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:47.323486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:47.323486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:47.323486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:47.323486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:47.323486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:47.323486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:47.323486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:47.323486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:47.323486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:47.323486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:47.323486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:08:47.323486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:08:47.323486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:47.324399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:47.325454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:08:47.325454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:08:47.325454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:08:47.325454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:08:47.325454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:08:47.325454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:08:47.325454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:47.325454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:47.325454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 299/601] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:47.332308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4202f06-1984-4687-be83-99866cf23ebb None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:47.448904 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:47.454414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4202f06-1984-4687-be83-99866cf23ebb None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:47.494902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5869d77-c0eb-425a-91c0-7c0bba6906a7 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:47.495368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37793b56-a8dd-4001-ac5a-649e23d30af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:47.496019 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37793b56-a8dd-4001-ac5a-649e23d30af0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:47.496558 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37793b56-a8dd-4001-ac5a-649e23d30af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:47.496835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37793b56-a8dd-4001-ac5a-649e23d30af0 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:47.497525 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37793b56-a8dd-4001-ac5a-649e23d30af0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:47.497832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37793b56-a8dd-4001-ac5a-649e23d30af0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:47.500272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37793b56-a8dd-4001-ac5a-649e23d30af0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:47.518862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 300/602] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:08:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:47.520948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37793b56-a8dd-4001-ac5a-649e23d30af0 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:47.580231 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58efe82-1b7a-4f12-bcab-633e0920d346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:47.580861 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58efe82-1b7a-4f12-bcab-633e0920d346 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:47.581145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58efe82-1b7a-4f12-bcab-633e0920d346 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:47.581423 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58efe82-1b7a-4f12-bcab-633e0920d346 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:47.582078 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58efe82-1b7a-4f12-bcab-633e0920d346 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:47.582388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58efe82-1b7a-4f12-bcab-633e0920d346 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:47.583797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58efe82-1b7a-4f12-bcab-633e0920d346 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:47.609999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 303/603] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:47.615150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58efe82-1b7a-4f12-bcab-633e0920d346 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:47.635478 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37793b56-a8dd-4001-ac5a-649e23d30af0 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:47.676388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e0ab8-fe40-4c0e-b3cf-0acd40f3198d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:47.677564 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e0ab8-fe40-4c0e-b3cf-0acd40f3198d None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:47.678009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e0ab8-fe40-4c0e-b3cf-0acd40f3198d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:47.678587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e0ab8-fe40-4c0e-b3cf-0acd40f3198d None admin] PATH_INFO: `/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/6a889616f86346f48b7573969174ee57/roles/910821ade2904d23b75fa69e040fb82d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:47.695642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e0ab8-fe40-4c0e-b3cf-0acd40f3198d None admin] RBAC: Authorizing `identity:create_grant(project_id=314d3d13ba2e4a0f93f5d3f34c50a96f, user_id=6a889616f86346f48b7573969174ee57, role_id=910821ade2904d23b75fa69e040fb82d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:47.697288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e0ab8-fe40-4c0e-b3cf-0acd40f3198d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:47.727646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 301/604] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:08:47 2022] PUT /identity/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/6a889616f86346f48b7573969174ee57/roles/910821ade2904d23b75fa69e040fb82d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:08:47.789877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:47.814149 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58efe82-1b7a-4f12-bcab-633e0920d346 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:47.856391 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80389d7-9003-41d9-9f3a-a9481a903583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:47.857039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80389d7-9003-41d9-9f3a-a9481a903583 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:47.857320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80389d7-9003-41d9-9f3a-a9481a903583 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:47.857591 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80389d7-9003-41d9-9f3a-a9481a903583 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:47.865606 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80389d7-9003-41d9-9f3a-a9481a903583 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:47.867180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80389d7-9003-41d9-9f3a-a9481a903583 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:47.880101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-f80389d7-9003-41d9-9f3a-a9481a903583 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:47.880101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:47.880101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:47.880101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:47.880101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:47.880101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:47.880101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:47.880101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:47.880101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:47.880101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:47.880101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:47.880101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:47.880101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:08:47.880101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:08:47.880101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:47.881307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:47.882457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:08:47.882457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:08:47.882457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:08:47.882457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:08:47.882457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:08:47.882457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:08:47.882457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:47.882457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:47.882457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 304/605] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:08:47.889750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e0ab8-fe40-4c0e-b3cf-0acd40f3198d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:47.906261 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:47.958252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3ad6c-7f10-4677-90c9-c0154f198394 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:47.958887 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3ad6c-7f10-4677-90c9-c0154f198394 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:47.959363 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3ad6c-7f10-4677-90c9-c0154f198394 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:47.961925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 305/606] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:47.967823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba31f4-b5be-4f93-acb4-6e9f6495a8e3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:47.968444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba31f4-b5be-4f93-acb4-6e9f6495a8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:47.968910 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba31f4-b5be-4f93-acb4-6e9f6495a8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:48.001313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f7ba31f4-b5be-4f93-acb4-6e9f6495a8e3 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:48.002911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f7ba31f4-b5be-4f93-acb4-6e9f6495a8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:48.035787 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e0ab8-fe40-4c0e-b3cf-0acd40f3198d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:48.083311 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7cc4e3-bc71-4c3a-bd49-c4155673284a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:48.083875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cc4e3-bc71-4c3a-bd49-c4155673284a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:48.084156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cc4e3-bc71-4c3a-bd49-c4155673284a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:48.084428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cc4e3-bc71-4c3a-bd49-c4155673284a None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:48.085114 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7cc4e3-bc71-4c3a-bd49-c4155673284a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:48.085413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7cc4e3-bc71-4c3a-bd49-c4155673284a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:48.086890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7cc4e3-bc71-4c3a-bd49-c4155673284a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:48.091927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 306/607] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:08:48.103732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 302/608] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:08:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:48.109231 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba31f4-b5be-4f93-acb4-6e9f6495a8e3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:48.203401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:48.232555 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba31f4-b5be-4f93-acb4-6e9f6495a8e3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:48.264584 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19c250-35aa-45ae-ad3f-227be466e444 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:48.264867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19c250-35aa-45ae-ad3f-227be466e444 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:48.265138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19c250-35aa-45ae-ad3f-227be466e444 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:48.266683 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 303/609] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:48.271632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0e8ea-0905-4a5b-8a56-d413fd74d1b6 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:48.271918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0e8ea-0905-4a5b-8a56-d413fd74d1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:48.272263 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0e8ea-0905-4a5b-8a56-d413fd74d1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:48.290906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-63f0e8ea-0905-4a5b-8a56-d413fd74d1b6 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:48.292117 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-63f0e8ea-0905-4a5b-8a56-d413fd74d1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:48.301114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b9c64-2696-4941-a586-72208be0e495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:48.301899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b9c64-2696-4941-a586-72208be0e495 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:48.302283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b9c64-2696-4941-a586-72208be0e495 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:48.302658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b9c64-2696-4941-a586-72208be0e495 None admin] PATH_INFO: `/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/2e71f9c1626f4f58916a5dd5c5987ccb/roles/910821ade2904d23b75fa69e040fb82d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:48.330310 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b9c64-2696-4941-a586-72208be0e495 None admin] RBAC: Authorizing `identity:create_grant(project_id=314d3d13ba2e4a0f93f5d3f34c50a96f, user_id=2e71f9c1626f4f58916a5dd5c5987ccb, role_id=910821ade2904d23b75fa69e040fb82d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:48.331938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b9c64-2696-4941-a586-72208be0e495 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:48.357806 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:48.361339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 307/610] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:08:48 2022] PUT /identity/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/2e71f9c1626f4f58916a5dd5c5987ccb/roles/910821ade2904d23b75fa69e040fb82d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:08:48.364333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 304/611] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:08:48.416918 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b9c64-2696-4941-a586-72208be0e495 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:48.524384 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b9c64-2696-4941-a586-72208be0e495 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:48.552471 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:48.562643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe4667-8dbb-4143-ad17-56e99885c0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:48.563330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe4667-8dbb-4143-ad17-56e99885c0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:48.563991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe4667-8dbb-4143-ad17-56e99885c0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:48.564339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe4667-8dbb-4143-ad17-56e99885c0d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:48.576364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe4667-8dbb-4143-ad17-56e99885c0d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:48.581255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-6abe4667-8dbb-4143-ad17-56e99885c0d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:48.581255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:48.581255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:48.581255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:48.581255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:48.581255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:48.581255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:48.581255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:48.581255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:48.581255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:48.581255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:48.581255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:48.581255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:48.581255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:48.581255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:48.581255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:48.582194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:48.582194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:48.582194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:48.582194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:48.582194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:48.582194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:48.582194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:48.582194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:48.582194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:48.582194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:48.582194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:48.582194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:48.582194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:48.582194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:48.582194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:48.582194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:48.583046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:48.583046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:48.583046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:48.583046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:48.583046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:48.583046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:48.583046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:48.583046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:48.583046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:48.583046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:48.583046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:48.583046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:48.583046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 308/612] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:48.588773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0e8ea-0905-4a5b-8a56-d413fd74d1b6 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:48.639850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe4667-8dbb-4143-ad17-56e99885c0d4 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:48.756655 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0e8ea-0905-4a5b-8a56-d413fd74d1b6 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:48.780946 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe4667-8dbb-4143-ad17-56e99885c0d4 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:48.822063 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f26eb-25ae-42a1-886d-43420f65650a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:48.822980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f26eb-25ae-42a1-886d-43420f65650a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:48.823448 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f26eb-25ae-42a1-886d-43420f65650a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:48.823922 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f26eb-25ae-42a1-886d-43420f65650a None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:48.824765 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f26eb-25ae-42a1-886d-43420f65650a None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:48.825273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f26eb-25ae-42a1-886d-43420f65650a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:48.826836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f26eb-25ae-42a1-886d-43420f65650a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:48.839561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 305/613] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:08:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:48.842204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97b99b-2313-4341-a56c-a84e8a3ca325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:48.843056 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97b99b-2313-4341-a56c-a84e8a3ca325 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:48.843565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97b99b-2313-4341-a56c-a84e8a3ca325 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:48.844032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97b99b-2313-4341-a56c-a84e8a3ca325 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:48.844896 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f26eb-25ae-42a1-886d-43420f65650a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:48.856017 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97b99b-2313-4341-a56c-a84e8a3ca325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:48.861204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-2e97b99b-2313-4341-a56c-a84e8a3ca325 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:48.861204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:48.861204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:48.861204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:48.861204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:48.861204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:48.861204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:48.861204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:48.861204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:48.861204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:48.861204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:48.861204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:48.861204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:48.861204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:48.861204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:48.861204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:48.862166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:48.862166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:48.862166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:48.862166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:48.862166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:48.862166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:48.862166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:48.862166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:48.862166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:48.862166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:48.862166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:48.862166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:48.862166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:48.862166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:48.862166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:48.862166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:48.863030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:48.863030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:48.863030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:48.863030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:48.863030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:48.863030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:48.863030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:48.863030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:48.863030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:48.863030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:48.863030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:48.863030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:48.865771 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 309/614] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:08:48.871956 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97b99b-2313-4341-a56c-a84e8a3ca325 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:48.951659 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:48.957823 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:49.010088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f26eb-25ae-42a1-886d-43420f65650a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:49.047058 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97b99b-2313-4341-a56c-a84e8a3ca325 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:49.073625 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18acdcc-ed30-4f6d-a1ba-3dd69dd63ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:49.074274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18acdcc-ed30-4f6d-a1ba-3dd69dd63ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:49.074616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18acdcc-ed30-4f6d-a1ba-3dd69dd63ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:49.074892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18acdcc-ed30-4f6d-a1ba-3dd69dd63ee3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:49.092875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18acdcc-ed30-4f6d-a1ba-3dd69dd63ee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:49.094340 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18acdcc-ed30-4f6d-a1ba-3dd69dd63ee3 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:49.116253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-e18acdcc-ed30-4f6d-a1ba-3dd69dd63ee3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:49.116253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:49.116253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:49.116253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:49.116253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:49.116253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:49.116253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:49.116253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:49.116253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:49.116253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:49.116253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:49.116253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:49.116253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:49.116253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:49.116253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:49.116253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:49.117834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:49.118737 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:49.118737 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:49.118737 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:49.118737 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 306/615] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:08:49.121204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17e80c-718a-40dd-a18f-2b35e18c6126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:49.121987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17e80c-718a-40dd-a18f-2b35e18c6126 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:49.122408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17e80c-718a-40dd-a18f-2b35e18c6126 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:49.122820 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17e80c-718a-40dd-a18f-2b35e18c6126 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:49.123682 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17e80c-718a-40dd-a18f-2b35e18c6126 None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:49.123808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18acdcc-ed30-4f6d-a1ba-3dd69dd63ee3 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:49.124234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17e80c-718a-40dd-a18f-2b35e18c6126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:49.125732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17e80c-718a-40dd-a18f-2b35e18c6126 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:49.142596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 310/616] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:08:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:49.148695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17e80c-718a-40dd-a18f-2b35e18c6126 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:49.256200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17e80c-718a-40dd-a18f-2b35e18c6126 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:49.263840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:49.297608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a074f-3eb7-4b51-8bde-628443a81d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:49.298743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a074f-3eb7-4b51-8bde-628443a81d78 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:49.300231 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a074f-3eb7-4b51-8bde-628443a81d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:49.300854 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a074f-3eb7-4b51-8bde-628443a81d78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:49.314765 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18acdcc-ed30-4f6d-a1ba-3dd69dd63ee3 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:49.326059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a074f-3eb7-4b51-8bde-628443a81d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:49.327785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a074f-3eb7-4b51-8bde-628443a81d78 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:49.341951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:49.354848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-3f7a074f-3eb7-4b51-8bde-628443a81d78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:49.354848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:49.354848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:49.354848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:49.354848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:49.354848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:49.354848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:49.354848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:49.354848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:49.354848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:49.354848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:49.354848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:49.354848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:49.354848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:49.354848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:49.354848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:49.355840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:49.356749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:49.356749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:49.356749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:49.362727 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 311/617] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:49.368271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a074f-3eb7-4b51-8bde-628443a81d78 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:49.391612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e8e82-1c83-46a5-9d61-d534a55a9745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:49.392461 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e8e82-1c83-46a5-9d61-d534a55a9745 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:49.392571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e8e82-1c83-46a5-9d61-d534a55a9745 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:49.392782 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e8e82-1c83-46a5-9d61-d534a55a9745 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:49.393431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e8e82-1c83-46a5-9d61-d534a55a9745 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:49.393734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e8e82-1c83-46a5-9d61-d534a55a9745 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:49.395105 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e8e82-1c83-46a5-9d61-d534a55a9745 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:49.423719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 307/618] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:49.429080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e8e82-1c83-46a5-9d61-d534a55a9745 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:49.531645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a074f-3eb7-4b51-8bde-628443a81d78 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:49.596449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87246a49-62cf-48b3-b321-80e8fcc2e1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:49.598259 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87246a49-62cf-48b3-b321-80e8fcc2e1fc None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:49.598947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87246a49-62cf-48b3-b321-80e8fcc2e1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:49.600639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87246a49-62cf-48b3-b321-80e8fcc2e1fc None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:49.602087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87246a49-62cf-48b3-b321-80e8fcc2e1fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:49.602817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87246a49-62cf-48b3-b321-80e8fcc2e1fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:49.605230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87246a49-62cf-48b3-b321-80e8fcc2e1fc None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:49.613078 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e8e82-1c83-46a5-9d61-d534a55a9745 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:49.631402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 312/619] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:49.638008 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87246a49-62cf-48b3-b321-80e8fcc2e1fc None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:49.667676 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:49.686384 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbfcde9-bd2f-4c0c-819a-928684903fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:49.688338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfcde9-bd2f-4c0c-819a-928684903fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:49.689495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfcde9-bd2f-4c0c-819a-928684903fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:49.690323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfcde9-bd2f-4c0c-819a-928684903fe1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:49.706852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbfcde9-bd2f-4c0c-819a-928684903fe1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:49.708888 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbfcde9-bd2f-4c0c-819a-928684903fe1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:49.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-acbfcde9-bd2f-4c0c-819a-928684903fe1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 12:08:49.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:49.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:49.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:49.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:49.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:49.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:49.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:49.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:49.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:49.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:49.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:49.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:08:49.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:08:49.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:08:49.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:49.735195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:49.736532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:49.736532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:49.736532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:08:49.736532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:08:49.736532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:08:49.736532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:08:49.736532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:08:49.736532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:08:49.736532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 12:08:49.736532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:49.741067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 308/620] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:08:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:49.745427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbfcde9-bd2f-4c0c-819a-928684903fe1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:49.800682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:49.816372 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87246a49-62cf-48b3-b321-80e8fcc2e1fc None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:49.855966 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc93bd2-1296-43a6-9693-dbeb130a48d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:49.856972 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc93bd2-1296-43a6-9693-dbeb130a48d4 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:49.857465 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc93bd2-1296-43a6-9693-dbeb130a48d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:49.857943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc93bd2-1296-43a6-9693-dbeb130a48d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:49.865800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc93bd2-1296-43a6-9693-dbeb130a48d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:49.868029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc93bd2-1296-43a6-9693-dbeb130a48d4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:49.880258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-acc93bd2-1296-43a6-9693-dbeb130a48d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:08:49.880258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:49.880258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:49.880258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:49.880258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:49.880258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:49.880258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:49.880258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:49.880258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:49.880258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:49.880258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:49.880258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:49.880258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:08:49.880258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:08:49.880258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:08:49.880258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:49.881252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:49.882224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:49.882224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:49.882224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:08:49.882224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:08:49.882224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:08:49.882224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:08:49.882224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:08:49.882224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:08:49.882224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:08:49.882224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:49.884974 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 313/621] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Oct 6 12:08:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:08:49.890710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54aa71-92ca-4acc-90a6-a56bdc32908f None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:49.891615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54aa71-92ca-4acc-90a6-a56bdc32908f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:49.892298 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54aa71-92ca-4acc-90a6-a56bdc32908f None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:49.893920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbfcde9-bd2f-4c0c-819a-928684903fe1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:49.895278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 314/622] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:49.896234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54aa71-92ca-4acc-90a6-a56bdc32908f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:49.942144 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b12c4-b81c-4e82-bf7f-dc56996a24ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:49.942762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b12c4-b81c-4e82-bf7f-dc56996a24ae None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:49.943081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b12c4-b81c-4e82-bf7f-dc56996a24ae None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:49.943306 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b12c4-b81c-4e82-bf7f-dc56996a24ae None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:49.944357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b12c4-b81c-4e82-bf7f-dc56996a24ae None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:49.944831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b12c4-b81c-4e82-bf7f-dc56996a24ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:49.946730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b12c4-b81c-4e82-bf7f-dc56996a24ae None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:49.968220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 309/623] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Oct 6 12:08:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:49.969698 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d5790-bb7c-4d6e-a20b-0987d9f16c63 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:49.970642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d5790-bb7c-4d6e-a20b-0987d9f16c63 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:49.971197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d5790-bb7c-4d6e-a20b-0987d9f16c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:49.991108 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:49.995316 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-b39d5790-bb7c-4d6e-a20b-0987d9f16c63 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:49.997132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b39d5790-bb7c-4d6e-a20b-0987d9f16c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:50.021942 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54aa71-92ca-4acc-90a6-a56bdc32908f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:50.063536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bebda-a8ac-4548-a472-459ac343c2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:50.064226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bebda-a8ac-4548-a472-459ac343c2cc None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:50.064599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bebda-a8ac-4548-a472-459ac343c2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:50.064966 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bebda-a8ac-4548-a472-459ac343c2cc None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:50.065846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bebda-a8ac-4548-a472-459ac343c2cc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:50.066248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bebda-a8ac-4548-a472-459ac343c2cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:50.067885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bebda-a8ac-4548-a472-459ac343c2cc None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:50.084630 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 310/624] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:50.085725 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d5790-bb7c-4d6e-a20b-0987d9f16c63 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:50.095438 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 315/625] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Oct 6 12:08:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:50.101105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bebda-a8ac-4548-a472-459ac343c2cc None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:50.112019 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:50.237530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d5790-bb7c-4d6e-a20b-0987d9f16c63 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:50.302469 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bebda-a8ac-4548-a472-459ac343c2cc None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:50.306213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b388b-763b-4857-baf5-78e82c5d1373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:50.307017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b388b-763b-4857-baf5-78e82c5d1373 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:50.307436 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b388b-763b-4857-baf5-78e82c5d1373 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:50.307898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b388b-763b-4857-baf5-78e82c5d1373 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:50.323158 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b388b-763b-4857-baf5-78e82c5d1373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:50.325003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b388b-763b-4857-baf5-78e82c5d1373 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:50.341745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-5e2b388b-763b-4857-baf5-78e82c5d1373 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:50.341745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:50.341745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:50.341745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:50.341745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:50.341745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:50.341745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:50.341745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:50.341745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:50.341745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:50.341745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:50.341745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:50.341745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:50.341745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:50.341745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:50.341745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:50.342780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:50.343722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:50.343722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:50.343722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:50.346631 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 311/626] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:50.353813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b388b-763b-4857-baf5-78e82c5d1373 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:50.395640 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c603c8c-c515-4f72-9e84-e8aea71908ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:50.396257 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c603c8c-c515-4f72-9e84-e8aea71908ff None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:50.396533 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c603c8c-c515-4f72-9e84-e8aea71908ff None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:50.396803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c603c8c-c515-4f72-9e84-e8aea71908ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:50.403896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:50.474433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:50.478782 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c603c8c-c515-4f72-9e84-e8aea71908ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:50.480283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c603c8c-c515-4f72-9e84-e8aea71908ff None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:50.511145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b388b-763b-4857-baf5-78e82c5d1373 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:50.553318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfda250-629b-4839-8690-f3276a3418a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:50.553936 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfda250-629b-4839-8690-f3276a3418a2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:50.554218 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfda250-629b-4839-8690-f3276a3418a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:50.554544 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfda250-629b-4839-8690-f3276a3418a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:50.555199 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfda250-629b-4839-8690-f3276a3418a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:50.555499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfda250-629b-4839-8690-f3276a3418a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:50.556963 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfda250-629b-4839-8690-f3276a3418a2 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:50.569846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 312/627] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:50.575244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfda250-629b-4839-8690-f3276a3418a2 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:50.592371 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 316/628] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:50.593438 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a63670-b084-4698-b0dc-7be410cfa2c7 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:50.593721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a63670-b084-4698-b0dc-7be410cfa2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:50.594027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a63670-b084-4698-b0dc-7be410cfa2c7 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:50.599487 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 317/629] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:50.599487 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a63670-b084-4698-b0dc-7be410cfa2c7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:50.640375 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:50.689184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfda250-629b-4839-8690-f3276a3418a2 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:50.720878 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a63670-b084-4698-b0dc-7be410cfa2c7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:50.724647 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:50.730620 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5900e-1f79-45b6-9617-636a622fff02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:50.731404 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5900e-1f79-45b6-9617-636a622fff02 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:50.731720 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5900e-1f79-45b6-9617-636a622fff02 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:50.731996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5900e-1f79-45b6-9617-636a622fff02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:50.743348 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5900e-1f79-45b6-9617-636a622fff02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:50.748626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-abb5900e-1f79-45b6-9617-636a622fff02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:50.748626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:50.748626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:50.748626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:50.748626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:50.748626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:50.748626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:50.748626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:50.748626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:50.748626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:50.748626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:50.748626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:50.748626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:50.748626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:50.748626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:50.748626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:50.749587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:50.749587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:50.749587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:50.749587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:50.749587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:50.749587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:50.749587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:50.749587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:50.749587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:50.749587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:50.749587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:50.749587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:50.749587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:50.749587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:50.749587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:50.749587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:50.751213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:50.751213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:50.751213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:50.751213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:50.751213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:50.751213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:50.751213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:50.751213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:50.751213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:50.751213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:50.751213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:50.751213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:50.751213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 313/630] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:50.751986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5900e-1f79-45b6-9617-636a622fff02 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:50.762925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d890d00-e911-450a-8fc9-085b035b1ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:50.763389 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d890d00-e911-450a-8fc9-085b035b1ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:50.763696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d890d00-e911-450a-8fc9-085b035b1ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:50.763966 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d890d00-e911-450a-8fc9-085b035b1ca4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:50.771353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d890d00-e911-450a-8fc9-085b035b1ca4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:50.772813 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d890d00-e911-450a-8fc9-085b035b1ca4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:50.788126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-1d890d00-e911-450a-8fc9-085b035b1ca4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:50.788126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:50.788126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:50.788126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:50.788126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:50.788126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:50.788126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:50.788126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:50.788126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:50.788126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:50.788126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:50.788126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:50.788126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:08:50.788126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:08:50.788126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:50.789178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:50.790626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:08:50.790626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:08:50.790626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:08:50.790626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:08:50.790626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:08:50.790626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:08:50.790626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:50.790626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:50.790626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 318/631] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:50.792068 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a35e1-7d33-457b-9a8e-90a66369b66f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:50.792351 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a35e1-7d33-457b-9a8e-90a66369b66f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:50.792619 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a35e1-7d33-457b-9a8e-90a66369b66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:50.812291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-7d5a35e1-7d33-457b-9a8e-90a66369b66f None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:50.813515 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7d5a35e1-7d33-457b-9a8e-90a66369b66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:50.876035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5900e-1f79-45b6-9617-636a622fff02 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:50.890681 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 319/632] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:50.891547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a35e1-7d33-457b-9a8e-90a66369b66f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:50.925643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c7de1-bdb0-4189-bbc1-90a9d123f2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:50.927134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c7de1-bdb0-4189-bbc1-90a9d123f2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:50.927530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c7de1-bdb0-4189-bbc1-90a9d123f2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:50.927915 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c7de1-bdb0-4189-bbc1-90a9d123f2c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:50.932899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:50.939089 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c7de1-bdb0-4189-bbc1-90a9d123f2c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:50.940718 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c7de1-bdb0-4189-bbc1-90a9d123f2c1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:50.956303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-839c7de1-bdb0-4189-bbc1-90a9d123f2c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:50.956303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:50.956303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:50.956303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:50.956303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:50.956303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:50.956303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:50.956303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:50.956303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:50.956303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:50.956303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:50.956303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:50.956303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:50.956303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:50.956303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:50.956303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:50.957500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:50.958664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:50.958664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:50.958664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:50.959042 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 314/633] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:50.959843 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c7de1-bdb0-4189-bbc1-90a9d123f2c1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:51.006244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:51.020897 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a35e1-7d33-457b-9a8e-90a66369b66f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:51.056694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00838eaa-cf47-4383-b427-ccbae26afa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:51.057388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00838eaa-cf47-4383-b427-ccbae26afa75 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:51.057677 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00838eaa-cf47-4383-b427-ccbae26afa75 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:51.057948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00838eaa-cf47-4383-b427-ccbae26afa75 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:51.058680 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00838eaa-cf47-4383-b427-ccbae26afa75 None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:51.058918 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00838eaa-cf47-4383-b427-ccbae26afa75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:51.060314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00838eaa-cf47-4383-b427-ccbae26afa75 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:51.064503 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c7de1-bdb0-4189-bbc1-90a9d123f2c1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:51.073293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 320/634] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:08:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:51.073293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73837e-18f6-4d49-80bb-392c1aa07c52 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:51.073293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73837e-18f6-4d49-80bb-392c1aa07c52 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:51.073293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73837e-18f6-4d49-80bb-392c1aa07c52 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:51.076113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 321/635] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:51.076921 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e73837e-18f6-4d49-80bb-392c1aa07c52 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:51.102478 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d65d2d-fc14-4551-9ae3-5cc048312bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:51.103094 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d65d2d-fc14-4551-9ae3-5cc048312bdf None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:51.103375 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d65d2d-fc14-4551-9ae3-5cc048312bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:51.103684 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d65d2d-fc14-4551-9ae3-5cc048312bdf None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:51.104369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d65d2d-fc14-4551-9ae3-5cc048312bdf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:51.104676 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d65d2d-fc14-4551-9ae3-5cc048312bdf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:51.106102 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d65d2d-fc14-4551-9ae3-5cc048312bdf None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:51.118780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 315/636] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:08:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:51.119617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d65d2d-fc14-4551-9ae3-5cc048312bdf None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:51.180601 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:51.195334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e73837e-18f6-4d49-80bb-392c1aa07c52 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:51.229262 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff750b3-3a81-4744-b15f-b4ff9ea1065b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:51.229653 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d65d2d-fc14-4551-9ae3-5cc048312bdf None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:51.229868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff750b3-3a81-4744-b15f-b4ff9ea1065b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:51.230150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff750b3-3a81-4744-b15f-b4ff9ea1065b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:51.230424 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff750b3-3a81-4744-b15f-b4ff9ea1065b None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:51.231061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff750b3-3a81-4744-b15f-b4ff9ea1065b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:51.231364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff750b3-3a81-4744-b15f-b4ff9ea1065b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:51.232169 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:51.232864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff750b3-3a81-4744-b15f-b4ff9ea1065b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:51.247207 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 322/637] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:51.250702 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cc0f8-489c-4479-ba57-28c70ffb93bf None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:51.250702 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cc0f8-489c-4479-ba57-28c70ffb93bf None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:51.250702 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cc0f8-489c-4479-ba57-28c70ffb93bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:51.266836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89891f54-ff40-4fc9-8285-8fd43197078b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:51.267768 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89891f54-ff40-4fc9-8285-8fd43197078b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:51.268024 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-eb1cc0f8-489c-4479-ba57-28c70ffb93bf None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:51.268247 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89891f54-ff40-4fc9-8285-8fd43197078b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:51.268497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89891f54-ff40-4fc9-8285-8fd43197078b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:51.269708 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-eb1cc0f8-489c-4479-ba57-28c70ffb93bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:51.280573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89891f54-ff40-4fc9-8285-8fd43197078b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:51.282013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89891f54-ff40-4fc9-8285-8fd43197078b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:51.297509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-89891f54-ff40-4fc9-8285-8fd43197078b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:51.297509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:51.297509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:51.297509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:51.297509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:51.297509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:51.297509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:51.297509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:51.297509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:51.297509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:51.297509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:51.297509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:51.297509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:51.297509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:51.297509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:51.297509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:51.298734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:51.299947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:51.299947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:51.299947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:51.300189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 316/638] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:51.301034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89891f54-ff40-4fc9-8285-8fd43197078b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:51.342733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 323/639] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:51.343609 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cc0f8-489c-4479-ba57-28c70ffb93bf None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:51.411315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89891f54-ff40-4fc9-8285-8fd43197078b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:51.437181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:51.447528 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a114bc-8e9a-46c9-918e-7d7c65daabbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:51.448208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a114bc-8e9a-46c9-918e-7d7c65daabbd None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:51.448490 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a114bc-8e9a-46c9-918e-7d7c65daabbd None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:51.448764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a114bc-8e9a-46c9-918e-7d7c65daabbd None admin] PATH_INFO: `/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/3deb4b248f884c19bf1e444f2fe0923a/roles/910821ade2904d23b75fa69e040fb82d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:51.465179 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a114bc-8e9a-46c9-918e-7d7c65daabbd None admin] RBAC: Authorizing `identity:create_grant(project_id=314d3d13ba2e4a0f93f5d3f34c50a96f, user_id=3deb4b248f884c19bf1e444f2fe0923a, role_id=910821ade2904d23b75fa69e040fb82d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:51.465702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a114bc-8e9a-46c9-918e-7d7c65daabbd None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:51.467198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cc0f8-489c-4479-ba57-28c70ffb93bf None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:51.480030 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 317/640] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:08:51 2022] PUT /identity/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/3deb4b248f884c19bf1e444f2fe0923a/roles/910821ade2904d23b75fa69e040fb82d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:08:51.480842 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a114bc-8e9a-46c9-918e-7d7c65daabbd None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:51.497255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:51.519680 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd0c06-95ba-48de-851d-ab28834c206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:51.520727 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd0c06-95ba-48de-851d-ab28834c206d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:51.521280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd0c06-95ba-48de-851d-ab28834c206d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:51.521596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd0c06-95ba-48de-851d-ab28834c206d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:51.535735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd0c06-95ba-48de-851d-ab28834c206d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:51.540978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-33fd0c06-95ba-48de-851d-ab28834c206d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:51.540978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:51.540978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:51.540978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:51.540978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:51.540978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:51.540978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:51.540978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:51.540978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:51.540978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:51.540978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:51.540978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:51.540978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:51.540978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:51.540978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:51.540978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:51.541994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:51.541994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:51.541994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:51.541994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:51.541994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:51.541994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:51.541994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:51.541994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:51.541994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:51.541994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:51.541994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:51.541994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:51.541994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:51.541994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:51.541994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:51.541994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:51.543773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:51.543773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:51.543773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:51.543773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:51.543773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:51.543773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:51.543773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:51.543773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:51.543773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:51.543773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:51.543773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:51.543773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:51.547300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 324/641] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:51.548171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd0c06-95ba-48de-851d-ab28834c206d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:51.629063 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a114bc-8e9a-46c9-918e-7d7c65daabbd None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:51.668688 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d369531-c2fe-4d39-a66d-fd212c1ce257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:51.669329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d369531-c2fe-4d39-a66d-fd212c1ce257 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:51.669608 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d369531-c2fe-4d39-a66d-fd212c1ce257 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:51.669877 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d369531-c2fe-4d39-a66d-fd212c1ce257 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:51.670193 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd0c06-95ba-48de-851d-ab28834c206d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:51.678014 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d369531-c2fe-4d39-a66d-fd212c1ce257 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:51.679543 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d369531-c2fe-4d39-a66d-fd212c1ce257 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:51.696028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-3d369531-c2fe-4d39-a66d-fd212c1ce257 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:51.696028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:51.696028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:51.696028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:51.696028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:51.696028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:51.696028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:51.696028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:51.696028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:51.696028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:51.696028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:51.696028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:51.696028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:08:51.696028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:08:51.696028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:51.697038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:51.697971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:08:51.697971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:08:51.697971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:08:51.697971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:08:51.697971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:08:51.697971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:08:51.697971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:51.697971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:51.697971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 318/642] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:51.697971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d369531-c2fe-4d39-a66d-fd212c1ce257 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:51.717570 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca519a1-415f-4038-9abf-20a88086b694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:51.718189 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca519a1-415f-4038-9abf-20a88086b694 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:51.718472 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca519a1-415f-4038-9abf-20a88086b694 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:51.718747 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca519a1-415f-4038-9abf-20a88086b694 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:51.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca519a1-415f-4038-9abf-20a88086b694 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:51.719731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca519a1-415f-4038-9abf-20a88086b694 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:51.721110 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca519a1-415f-4038-9abf-20a88086b694 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:51.728441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:51.734534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 325/643] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:51.735190 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca519a1-415f-4038-9abf-20a88086b694 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:51.767843 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:51.806327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d369531-c2fe-4d39-a66d-fd212c1ce257 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:51.837236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca519a1-415f-4038-9abf-20a88086b694 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:51.840971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a1696-0529-4ee5-97c4-ac0e50b1c568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:51.841578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a1696-0529-4ee5-97c4-ac0e50b1c568 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:51.841855 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a1696-0529-4ee5-97c4-ac0e50b1c568 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:51.842122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a1696-0529-4ee5-97c4-ac0e50b1c568 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:51.842788 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a1696-0529-4ee5-97c4-ac0e50b1c568 None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:51.843089 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a1696-0529-4ee5-97c4-ac0e50b1c568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:51.844681 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a1696-0529-4ee5-97c4-ac0e50b1c568 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:51.858383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 319/644] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:08:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:51.858383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a1696-0529-4ee5-97c4-ac0e50b1c568 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:51.876505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32595664-c634-493d-b4c6-67d999356cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:51.877139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32595664-c634-493d-b4c6-67d999356cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:51.877488 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32595664-c634-493d-b4c6-67d999356cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:51.877696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32595664-c634-493d-b4c6-67d999356cf0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:51.889279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32595664-c634-493d-b4c6-67d999356cf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:51.894585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-32595664-c634-493d-b4c6-67d999356cf0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:51.894585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:51.894585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:51.894585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:51.894585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:51.894585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:51.894585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:51.894585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:51.894585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:51.894585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:51.894585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:51.894585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:51.894585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:51.894585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:51.894585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:51.894585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:51.895565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:51.895565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:51.895565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:51.895565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:51.895565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:51.895565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:51.895565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:51.895565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:51.895565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:51.895565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:51.895565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:51.895565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:51.895565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:51.895565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:51.895565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:51.895565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:51.896592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:51.896592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:51.896592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:51.896592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:51.896592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:51.896592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:51.896592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:51.896592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:51.896592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:51.896592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:51.896592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:51.896592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:51.900114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 326/645] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:51.901108 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32595664-c634-493d-b4c6-67d999356cf0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:51.957875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a1696-0529-4ee5-97c4-ac0e50b1c568 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:51.962782 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:51.985231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:51.995287 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec789014-bb14-4f40-b2f9-0610ddfb1f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:51.996227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec789014-bb14-4f40-b2f9-0610ddfb1f11 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:51.996513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec789014-bb14-4f40-b2f9-0610ddfb1f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:51.996792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec789014-bb14-4f40-b2f9-0610ddfb1f11 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:51.997488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec789014-bb14-4f40-b2f9-0610ddfb1f11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:51.997798 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec789014-bb14-4f40-b2f9-0610ddfb1f11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:51.999765 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec789014-bb14-4f40-b2f9-0610ddfb1f11 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:52.011220 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32595664-c634-493d-b4c6-67d999356cf0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:52.017136 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 320/646] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:08:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:52.017136 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec789014-bb14-4f40-b2f9-0610ddfb1f11 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:52.047080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecc0d0-e7ea-41cc-be03-355ad01ea735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:52.047726 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecc0d0-e7ea-41cc-be03-355ad01ea735 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:52.048003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecc0d0-e7ea-41cc-be03-355ad01ea735 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:52.048382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecc0d0-e7ea-41cc-be03-355ad01ea735 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:52.056482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecc0d0-e7ea-41cc-be03-355ad01ea735 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:52.057923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecc0d0-e7ea-41cc-be03-355ad01ea735 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:52.069097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-3eecc0d0-e7ea-41cc-be03-355ad01ea735 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 12:08:52.069097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:52.069097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:52.069097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:52.069097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:52.069097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:52.069097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:52.069097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:52.069097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:52.069097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:52.069097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:52.069097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:52.069097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:08:52.069097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:08:52.069097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:08:52.069097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:52.070802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:52.072032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:52.072032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:52.072032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:08:52.072032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:08:52.072032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:08:52.072032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:08:52.072032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:08:52.072032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:08:52.072032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 12:08:52.072032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:52.072032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 327/647] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:52.072032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecc0d0-e7ea-41cc-be03-355ad01ea735 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:52.127390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec789014-bb14-4f40-b2f9-0610ddfb1f11 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:52.165095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b23c8-9e07-4919-bfa6-ab13cd9c1e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:52.165978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b23c8-9e07-4919-bfa6-ab13cd9c1e9f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:52.166450 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b23c8-9e07-4919-bfa6-ab13cd9c1e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:52.166995 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b23c8-9e07-4919-bfa6-ab13cd9c1e9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:52.172862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecc0d0-e7ea-41cc-be03-355ad01ea735 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:52.177633 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b23c8-9e07-4919-bfa6-ab13cd9c1e9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:52.179774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b23c8-9e07-4919-bfa6-ab13cd9c1e9f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:52.190250 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:52.197554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-191b23c8-9e07-4919-bfa6-ab13cd9c1e9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:52.197554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:52.197554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:52.197554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:52.197554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:52.197554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:52.197554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:52.197554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:52.197554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:52.197554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:52.197554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:52.197554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:52.197554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:52.197554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:52.197554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:52.197554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:52.198793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:52.200035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:52.200035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:52.200035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:52.201542 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 321/648] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:52.202592 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b23c8-9e07-4919-bfa6-ab13cd9c1e9f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:52.212992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d33b4e-2d13-42ff-a8e2-3d5d409873e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:52.213895 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d33b4e-2d13-42ff-a8e2-3d5d409873e1 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:52.214174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d33b4e-2d13-42ff-a8e2-3d5d409873e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:52.214439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d33b4e-2d13-42ff-a8e2-3d5d409873e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:52.215094 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d33b4e-2d13-42ff-a8e2-3d5d409873e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:52.215392 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d33b4e-2d13-42ff-a8e2-3d5d409873e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:52.216790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d33b4e-2d13-42ff-a8e2-3d5d409873e1 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:52.230511 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 328/649] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:08:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:52.230511 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d33b4e-2d13-42ff-a8e2-3d5d409873e1 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:52.258109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:52.319935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b23c8-9e07-4919-bfa6-ab13cd9c1e9f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:52.345804 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d33b4e-2d13-42ff-a8e2-3d5d409873e1 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:52.353462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ce95c-15b1-47df-88cb-f8dcb9c663be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:52.354072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ce95c-15b1-47df-88cb-f8dcb9c663be None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:52.354364 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ce95c-15b1-47df-88cb-f8dcb9c663be None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:52.354642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ce95c-15b1-47df-88cb-f8dcb9c663be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:52.355364 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ce95c-15b1-47df-88cb-f8dcb9c663be None admin] RBAC: Adding query filter params (role.id=910821ade2904d23b75fa69e040fb82d, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=5a46e63d34564d34b6b5752706375798) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:52.355722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ce95c-15b1-47df-88cb-f8dcb9c663be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:52.357141 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ce95c-15b1-47df-88cb-f8dcb9c663be None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:52.371536 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 322/650] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:08:52 2022] GET /identity/v3/role_assignments?user.id=5a46e63d34564d34b6b5752706375798&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=910821ade2904d23b75fa69e040fb82d => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:52.371536 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ce95c-15b1-47df-88cb-f8dcb9c663be None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:52.398296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b633e-2c10-423f-ae17-5806c02cd653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:52.399429 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b633e-2c10-423f-ae17-5806c02cd653 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:52.400430 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b633e-2c10-423f-ae17-5806c02cd653 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:52.400976 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b633e-2c10-423f-ae17-5806c02cd653 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:52.401979 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b633e-2c10-423f-ae17-5806c02cd653 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:52.402407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b633e-2c10-423f-ae17-5806c02cd653 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:52.404341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b633e-2c10-423f-ae17-5806c02cd653 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:52.426572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 329/651] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Oct 6 12:08:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:52.428436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b633e-2c10-423f-ae17-5806c02cd653 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:52.480473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:52.486457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ce95c-15b1-47df-88cb-f8dcb9c663be None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:52.490486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:52.527232 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7f420-dad6-437f-87eb-7ee3905df6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:52.528050 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7f420-dad6-437f-87eb-7ee3905df6aa None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:52.528435 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7f420-dad6-437f-87eb-7ee3905df6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:52.528854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7f420-dad6-437f-87eb-7ee3905df6aa None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:52.529703 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7f420-dad6-437f-87eb-7ee3905df6aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:52.530388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7f420-dad6-437f-87eb-7ee3905df6aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:52.531905 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7f420-dad6-437f-87eb-7ee3905df6aa None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:52.551208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 323/652] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:52.552346 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7f420-dad6-437f-87eb-7ee3905df6aa None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:52.554673 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b633e-2c10-423f-ae17-5806c02cd653 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:52.605368 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c83f4-9717-4732-899f-7dd7c0c47e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:52.606447 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c83f4-9717-4732-899f-7dd7c0c47e8c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:52.606898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c83f4-9717-4732-899f-7dd7c0c47e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:52.607725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c83f4-9717-4732-899f-7dd7c0c47e8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:52.619631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c83f4-9717-4732-899f-7dd7c0c47e8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:52.621085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c83f4-9717-4732-899f-7dd7c0c47e8c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:52.640787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-da9c83f4-9717-4732-899f-7dd7c0c47e8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:52.640787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:52.640787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:52.640787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:52.640787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:52.640787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:52.640787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:52.640787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:52.640787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:52.640787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:52.640787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:52.640787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:52.640787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:52.640787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:52.640787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:52.640787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:52.641837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:52.642825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:52.642825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:52.642825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:52.642825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 330/653] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:52.643065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c83f4-9717-4732-899f-7dd7c0c47e8c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:52.678365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7f420-dad6-437f-87eb-7ee3905df6aa None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:52.725927 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104b674-41ce-48a2-b1a3-0036b2c211d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:52.727130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104b674-41ce-48a2-b1a3-0036b2c211d7 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:52.727474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104b674-41ce-48a2-b1a3-0036b2c211d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:52.727807 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104b674-41ce-48a2-b1a3-0036b2c211d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:52.761218 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:52.765451 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:52.775174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c83f4-9717-4732-899f-7dd7c0c47e8c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:52.813801 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104b674-41ce-48a2-b1a3-0036b2c211d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:52.815734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104b674-41ce-48a2-b1a3-0036b2c211d7 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:52.817964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840296f-bb6c-4729-a739-9357cad0a561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:52.819470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840296f-bb6c-4729-a739-9357cad0a561 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:52.819909 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840296f-bb6c-4729-a739-9357cad0a561 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:52.820315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840296f-bb6c-4729-a739-9357cad0a561 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:52.831016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840296f-bb6c-4729-a739-9357cad0a561 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:52.833314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840296f-bb6c-4729-a739-9357cad0a561 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:52.849106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-c840296f-bb6c-4729-a739-9357cad0a561 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 06 12:08:52.849106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:52.849106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:52.849106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:52.849106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:52.849106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:52.849106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:52.849106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:52.849106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:52.849106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:52.849106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:52.849106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:52.849106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:08:52.849106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:08:52.849106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:08:52.849106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:52.850286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:52.852026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:52.852026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:52.852026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:08:52.852026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:08:52.852026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:08:52.852026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:08:52.852026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:08:52.852026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:08:52.852026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 06 12:08:52.852026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:52.852026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 331/654] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:52.852026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840296f-bb6c-4729-a739-9357cad0a561 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:52.943774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 324/655] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:52.944710 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104b674-41ce-48a2-b1a3-0036b2c211d7 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:52.976068 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840296f-bb6c-4729-a739-9357cad0a561 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:53.010136 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:53.021468 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9995499-7811-416e-9829-4e4aba7ef725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:53.022140 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9995499-7811-416e-9829-4e4aba7ef725 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:53.022544 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9995499-7811-416e-9829-4e4aba7ef725 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:53.022951 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9995499-7811-416e-9829-4e4aba7ef725 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:53.023817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9995499-7811-416e-9829-4e4aba7ef725 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:53.024303 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9995499-7811-416e-9829-4e4aba7ef725 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:53.025775 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9995499-7811-416e-9829-4e4aba7ef725 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:53.040396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 332/656] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:53.041351 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9995499-7811-416e-9829-4e4aba7ef725 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:53.055439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104b674-41ce-48a2-b1a3-0036b2c211d7 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:53.056523 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:53.093910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab4021-5e76-4cb1-a072-059b4f9260de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:53.094327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab4021-5e76-4cb1-a072-059b4f9260de None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:53.094616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab4021-5e76-4cb1-a072-059b4f9260de None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:53.094890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab4021-5e76-4cb1-a072-059b4f9260de None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:53.095795 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab4021-5e76-4cb1-a072-059b4f9260de None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:53.096032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab4021-5e76-4cb1-a072-059b4f9260de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:53.097532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab4021-5e76-4cb1-a072-059b4f9260de None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:53.121822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 325/657] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Oct 6 12:08:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:53.122875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab4021-5e76-4cb1-a072-059b4f9260de None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:53.161335 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9995499-7811-416e-9829-4e4aba7ef725 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:53.203982 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e715c0-480c-4a47-8000-bf431ef6eb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:53.205259 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e715c0-480c-4a47-8000-bf431ef6eb8a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:53.206510 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e715c0-480c-4a47-8000-bf431ef6eb8a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:53.207155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e715c0-480c-4a47-8000-bf431ef6eb8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:53.221472 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e715c0-480c-4a47-8000-bf431ef6eb8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:53.224135 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e715c0-480c-4a47-8000-bf431ef6eb8a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:53.246426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-e0e715c0-480c-4a47-8000-bf431ef6eb8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:53.246426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:53.246426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:53.246426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:53.246426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:53.246426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:53.246426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:53.246426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:53.246426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:53.246426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:53.246426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:53.246426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:53.246426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:53.246426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:53.246426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:53.246426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:53.247436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:53.248433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:53.248433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:53.248433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:53.255018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 333/658] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:53.255018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e715c0-480c-4a47-8000-bf431ef6eb8a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:53.261886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab4021-5e76-4cb1-a072-059b4f9260de None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:53.296541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:53.310349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90924e-87ae-49eb-b49b-4de6b45f1477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:53.311347 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90924e-87ae-49eb-b49b-4de6b45f1477 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:53.311877 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90924e-87ae-49eb-b49b-4de6b45f1477 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:53.312416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90924e-87ae-49eb-b49b-4de6b45f1477 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:53.320146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90924e-87ae-49eb-b49b-4de6b45f1477 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:53.321886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90924e-87ae-49eb-b49b-4de6b45f1477 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:53.322720 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:53.334682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-5c90924e-87ae-49eb-b49b-4de6b45f1477 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 12:08:53.334682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:53.334682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:53.334682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:53.334682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:53.334682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:53.334682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:53.334682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:53.334682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:53.334682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:53.334682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:53.334682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:53.334682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:08:53.334682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:08:53.334682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:08:53.334682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:53.335987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:53.337201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:53.337201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:53.337201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:08:53.337201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:08:53.337201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:08:53.337201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:08:53.337201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:08:53.337201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:08:53.337201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 12:08:53.337201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:53.343410 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 326/659] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:53.343410 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90924e-87ae-49eb-b49b-4de6b45f1477 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:53.378302 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e715c0-480c-4a47-8000-bf431ef6eb8a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:53.426643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44346393-fa60-4b99-a45e-d001f34b951f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:53.427544 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44346393-fa60-4b99-a45e-d001f34b951f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:53.427976 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44346393-fa60-4b99-a45e-d001f34b951f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:53.428416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44346393-fa60-4b99-a45e-d001f34b951f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:53.469849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90924e-87ae-49eb-b49b-4de6b45f1477 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:53.501445 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44346393-fa60-4b99-a45e-d001f34b951f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:53.503440 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44346393-fa60-4b99-a45e-d001f34b951f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:53.505820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b848f8-f948-4543-90e4-e237f94e5509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:53.506551 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b848f8-f948-4543-90e4-e237f94e5509 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:53.507689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b848f8-f948-4543-90e4-e237f94e5509 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:53.507689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b848f8-f948-4543-90e4-e237f94e5509 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:53.508137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b848f8-f948-4543-90e4-e237f94e5509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:53.508600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b848f8-f948-4543-90e4-e237f94e5509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:53.510107 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b848f8-f948-4543-90e4-e237f94e5509 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:53.527641 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 327/660] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:53.528641 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b848f8-f948-4543-90e4-e237f94e5509 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:53.569338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:53.574261 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:53.619443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 334/661] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:53.620723 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44346393-fa60-4b99-a45e-d001f34b951f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:53.639680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b848f8-f948-4543-90e4-e237f94e5509 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:53.677240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c4177-e36b-4612-8173-f9e436c89e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:53.678003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c4177-e36b-4612-8173-f9e436c89e72 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:53.678391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c4177-e36b-4612-8173-f9e436c89e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:53.678822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c4177-e36b-4612-8173-f9e436c89e72 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:53.680098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c4177-e36b-4612-8173-f9e436c89e72 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:53.680511 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c4177-e36b-4612-8173-f9e436c89e72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:53.682070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c4177-e36b-4612-8173-f9e436c89e72 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:53.705061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 328/662] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Oct 6 12:08:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:53.705061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c4177-e36b-4612-8173-f9e436c89e72 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:53.739029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44346393-fa60-4b99-a45e-d001f34b951f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:53.775736 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f96db4-a330-4d2b-a541-fc739f93e8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:53.776371 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f96db4-a330-4d2b-a541-fc739f93e8cc None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:53.776622 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f96db4-a330-4d2b-a541-fc739f93e8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:53.776889 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f96db4-a330-4d2b-a541-fc739f93e8cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:53.783815 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f96db4-a330-4d2b-a541-fc739f93e8cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:53.785235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f96db4-a330-4d2b-a541-fc739f93e8cc None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:53.798370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-55f96db4-a330-4d2b-a541-fc739f93e8cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:53.798370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:53.798370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:53.798370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:53.798370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:53.798370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:53.798370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:53.798370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:53.798370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:53.798370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:53.798370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:53.798370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:53.798370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:08:53.798370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:08:53.798370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:53.799334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:53.800328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:08:53.800328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:08:53.800328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:08:53.800328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:08:53.800328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:08:53.800328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:08:53.800328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:53.800328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:53.800328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 335/663] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:53.800852 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f96db4-a330-4d2b-a541-fc739f93e8cc None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:53.821886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:53.825514 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c4177-e36b-4612-8173-f9e436c89e72 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:53.826579 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:53.870900 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e26f5-9f83-4f1d-b614-0bd792116792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:53.871577 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e26f5-9f83-4f1d-b614-0bd792116792 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:53.872028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e26f5-9f83-4f1d-b614-0bd792116792 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:53.872343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e26f5-9f83-4f1d-b614-0bd792116792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:53.883977 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e26f5-9f83-4f1d-b614-0bd792116792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:53.885420 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e26f5-9f83-4f1d-b614-0bd792116792 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:53.900339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-298e26f5-9f83-4f1d-b614-0bd792116792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:53.900339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:53.900339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:53.900339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:53.900339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:53.900339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:53.900339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:53.900339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:53.900339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:53.900339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:53.900339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:53.900339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:53.900339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:53.900339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:53.900339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:53.900339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:53.901312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:53.902260 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:53.902260 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:53.902260 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:53.902956 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f96db4-a330-4d2b-a541-fc739f93e8cc None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:53.906163 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 329/664] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:53.907348 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ae08b-dea0-4c67-9f82-91ee0e7cb7e8 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:53.907667 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ae08b-dea0-4c67-9f82-91ee0e7cb7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:53.907949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ae08b-dea0-4c67-9f82-91ee0e7cb7e8 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:53.911547 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 330/665] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:53.911547 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ae08b-dea0-4c67-9f82-91ee0e7cb7e8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:53.943025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9dff2a-e3cd-4285-af76-ffdcee0d87e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:53.943656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9dff2a-e3cd-4285-af76-ffdcee0d87e8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:53.943934 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9dff2a-e3cd-4285-af76-ffdcee0d87e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:53.944205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9dff2a-e3cd-4285-af76-ffdcee0d87e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:54.014087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9dff2a-e3cd-4285-af76-ffdcee0d87e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:54.015570 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9dff2a-e3cd-4285-af76-ffdcee0d87e8 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:54.020475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ae08b-dea0-4c67-9f82-91ee0e7cb7e8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:54.040220 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:54.056873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9cfc6-c47e-490f-953c-89bb33219d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:54.057985 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9cfc6-c47e-490f-953c-89bb33219d11 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:54.058270 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9cfc6-c47e-490f-953c-89bb33219d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:54.058550 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9cfc6-c47e-490f-953c-89bb33219d11 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:54.059630 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9cfc6-c47e-490f-953c-89bb33219d11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:54.059945 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9cfc6-c47e-490f-953c-89bb33219d11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:54.061678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9cfc6-c47e-490f-953c-89bb33219d11 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:54.073803 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 331/666] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:08:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:54.074456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9cfc6-c47e-490f-953c-89bb33219d11 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:54.088858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:54.128411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 336/667] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:54.128411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae725ca6-e4f6-4053-846c-4b8b613753ef None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:54.128411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae725ca6-e4f6-4053-846c-4b8b613753ef None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:54.128411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae725ca6-e4f6-4053-846c-4b8b613753ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:54.144949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ae725ca6-e4f6-4053-846c-4b8b613753ef None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:54.146238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ae725ca6-e4f6-4053-846c-4b8b613753ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:54.182702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9cfc6-c47e-490f-953c-89bb33219d11 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:54.217221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb471d4-8d58-434d-95ad-9037dd2c0870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:54.217833 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb471d4-8d58-434d-95ad-9037dd2c0870 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:54.218116 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb471d4-8d58-434d-95ad-9037dd2c0870 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:54.218391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb471d4-8d58-434d-95ad-9037dd2c0870 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:54.219029 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb471d4-8d58-434d-95ad-9037dd2c0870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:54.219329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb471d4-8d58-434d-95ad-9037dd2c0870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:54.220725 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb471d4-8d58-434d-95ad-9037dd2c0870 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:54.227855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 337/668] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:54.229176 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae725ca6-e4f6-4053-846c-4b8b613753ef None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:54.238266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 332/669] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:54.238266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb471d4-8d58-434d-95ad-9037dd2c0870 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:54.300576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:54.327632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:54.357173 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae725ca6-e4f6-4053-846c-4b8b613753ef None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:54.371069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb471d4-8d58-434d-95ad-9037dd2c0870 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:54.393273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdd28a-ef15-4a9e-b06a-1e411a6bf97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:54.394189 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdd28a-ef15-4a9e-b06a-1e411a6bf97d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:54.394674 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdd28a-ef15-4a9e-b06a-1e411a6bf97d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:54.395168 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdd28a-ef15-4a9e-b06a-1e411a6bf97d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:54.396171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdd28a-ef15-4a9e-b06a-1e411a6bf97d None admin] RBAC: Adding query filter params (role.id=910821ade2904d23b75fa69e040fb82d, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=43e873f881e6497ea09f7f10f08e3bd4) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:54.396705 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdd28a-ef15-4a9e-b06a-1e411a6bf97d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:54.398879 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdd28a-ef15-4a9e-b06a-1e411a6bf97d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:54.406740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 338/670] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:08:54 2022] GET /identity/v3/role_assignments?user.id=43e873f881e6497ea09f7f10f08e3bd4&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=910821ade2904d23b75fa69e040fb82d => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:54.407901 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdd28a-ef15-4a9e-b06a-1e411a6bf97d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:54.420752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d4038-99fb-4593-b4e0-299d4c33916c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:54.424342 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d4038-99fb-4593-b4e0-299d4c33916c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:54.424946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d4038-99fb-4593-b4e0-299d4c33916c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:54.425429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d4038-99fb-4593-b4e0-299d4c33916c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:54.454369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d4038-99fb-4593-b4e0-299d4c33916c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:54.457246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d4038-99fb-4593-b4e0-299d4c33916c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:54.483958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-b51d4038-99fb-4593-b4e0-299d4c33916c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:54.483958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:54.483958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:54.483958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:54.483958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:54.483958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:54.483958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:54.483958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:54.483958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:54.483958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:54.483958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:54.483958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:54.483958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:54.483958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:54.483958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:54.483958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:54.484999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:54.485949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:54.485949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:54.485949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:54.495845 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 333/671] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:54.498039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d4038-99fb-4593-b4e0-299d4c33916c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:54.581155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdd28a-ef15-4a9e-b06a-1e411a6bf97d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:54.602403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:54.625479 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7039a-a8e7-465a-9501-8bb2d317722a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:54.626300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7039a-a8e7-465a-9501-8bb2d317722a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:54.626703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7039a-a8e7-465a-9501-8bb2d317722a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:54.627079 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7039a-a8e7-465a-9501-8bb2d317722a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:54.635359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7039a-a8e7-465a-9501-8bb2d317722a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:54.635841 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d4038-99fb-4593-b4e0-299d4c33916c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:54.637125 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7039a-a8e7-465a-9501-8bb2d317722a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:54.650099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-44e7039a-a8e7-465a-9501-8bb2d317722a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:54.650099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:54.650099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:54.650099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:54.650099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:54.650099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:54.650099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:54.650099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:54.650099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:54.650099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:54.650099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:54.650099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:54.650099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:08:54.650099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:08:54.650099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:54.651214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:54.652251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:08:54.652251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:08:54.652251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:08:54.652251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:08:54.652251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:08:54.652251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:08:54.652251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:54.652251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:54.652251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 339/672] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:54.652740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7039a-a8e7-465a-9501-8bb2d317722a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:54.668927 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:54.683607 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d39dea-35c2-4618-941c-0b7a84eed53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:54.684263 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d39dea-35c2-4618-941c-0b7a84eed53a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:54.684542 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d39dea-35c2-4618-941c-0b7a84eed53a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:54.684808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d39dea-35c2-4618-941c-0b7a84eed53a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:54.697764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d39dea-35c2-4618-941c-0b7a84eed53a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:54.704467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-60d39dea-35c2-4618-941c-0b7a84eed53a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:54.704467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:54.704467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:54.704467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:54.704467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:54.704467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:54.704467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:54.704467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:54.704467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:54.704467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:54.704467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:54.704467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:54.704467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:54.704467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:54.704467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:54.704467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:54.705454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:54.705454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:54.705454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:54.705454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:54.705454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:54.705454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:54.705454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:54.705454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:54.705454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:54.705454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:54.705454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:54.705454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:54.705454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:54.705454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:54.705454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:54.705454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:54.706314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:54.706314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:54.706314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:54.706314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:54.706314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:54.706314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:54.706314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:54.706314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:54.706314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:54.706314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:54.706314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:08:54.706314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:54.706314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 334/673] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:08:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:54.707650 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d39dea-35c2-4618-941c-0b7a84eed53a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:54.773780 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7039a-a8e7-465a-9501-8bb2d317722a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:54.809765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b79a71-77bb-4ffe-a3d2-86860ec9d12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:54.810706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b79a71-77bb-4ffe-a3d2-86860ec9d12c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:54.811362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b79a71-77bb-4ffe-a3d2-86860ec9d12c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:54.811667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b79a71-77bb-4ffe-a3d2-86860ec9d12c None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:54.812312 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b79a71-77bb-4ffe-a3d2-86860ec9d12c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:54.812666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b79a71-77bb-4ffe-a3d2-86860ec9d12c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:54.814030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b79a71-77bb-4ffe-a3d2-86860ec9d12c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:54.826915 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 340/674] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:54.827748 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b79a71-77bb-4ffe-a3d2-86860ec9d12c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:54.834526 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d39dea-35c2-4618-941c-0b7a84eed53a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:54.866787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:54.876111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385cbeb-bf49-4b53-963e-2bc4435eb11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:54.876721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385cbeb-bf49-4b53-963e-2bc4435eb11a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:54.877001 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385cbeb-bf49-4b53-963e-2bc4435eb11a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:54.877268 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385cbeb-bf49-4b53-963e-2bc4435eb11a None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:54.877951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385cbeb-bf49-4b53-963e-2bc4435eb11a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:54.878258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385cbeb-bf49-4b53-963e-2bc4435eb11a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:54.879709 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385cbeb-bf49-4b53-963e-2bc4435eb11a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:54.893472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 335/675] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:08:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:54.894678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0361caf-1fa9-490d-b327-45c213bff4a0 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:54.894960 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0361caf-1fa9-490d-b327-45c213bff4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:54.895230 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0361caf-1fa9-490d-b327-45c213bff4a0 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:54.896626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 336/676] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:54.901807 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0361caf-1fa9-490d-b327-45c213bff4a0 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:54.948474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b79a71-77bb-4ffe-a3d2-86860ec9d12c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:54.950385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:54.984868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115f797-d48c-4aea-9a41-7804263d2421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:54.985473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115f797-d48c-4aea-9a41-7804263d2421 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:54.985748 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115f797-d48c-4aea-9a41-7804263d2421 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:54.986017 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115f797-d48c-4aea-9a41-7804263d2421 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:54.986672 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115f797-d48c-4aea-9a41-7804263d2421 None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:54.986972 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115f797-d48c-4aea-9a41-7804263d2421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:54.988362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115f797-d48c-4aea-9a41-7804263d2421 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:55.001654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 341/677] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:08:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:55.001654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115f797-d48c-4aea-9a41-7804263d2421 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:55.016498 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0361caf-1fa9-490d-b327-45c213bff4a0 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:55.052879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c6bdc-ca51-402e-a1c6-011eaa54ea49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:55.054942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c6bdc-ca51-402e-a1c6-011eaa54ea49 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:55.055376 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c6bdc-ca51-402e-a1c6-011eaa54ea49 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:55.055981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c6bdc-ca51-402e-a1c6-011eaa54ea49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:55.063252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c6bdc-ca51-402e-a1c6-011eaa54ea49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:55.064817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c6bdc-ca51-402e-a1c6-011eaa54ea49 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:55.075875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-910c6bdc-ca51-402e-a1c6-011eaa54ea49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:55.075875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:55.075875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:55.075875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:55.075875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:55.075875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:55.075875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:55.075875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:55.075875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:55.075875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:55.075875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:55.075875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:55.075875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:08:55.075875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:08:55.075875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:55.076994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:55.078145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:08:55.078145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:08:55.078145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:08:55.078145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:08:55.078145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:08:55.078145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:08:55.078145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:55.078145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:55.078145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 337/678] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:55.078145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942d4fb-9594-4e84-b956-5d12451bcb3f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:55.078849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942d4fb-9594-4e84-b956-5d12451bcb3f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:55.078849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942d4fb-9594-4e84-b956-5d12451bcb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:55.096711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-1942d4fb-9594-4e84-b956-5d12451bcb3f None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:55.097812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1942d4fb-9594-4e84-b956-5d12451bcb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:55.115671 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:55.121818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115f797-d48c-4aea-9a41-7804263d2421 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:55.160559 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75b02f-998b-4479-bef4-6b8b4091feed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:55.161416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75b02f-998b-4479-bef4-6b8b4091feed None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:55.161868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75b02f-998b-4479-bef4-6b8b4091feed None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:55.162153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75b02f-998b-4479-bef4-6b8b4091feed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:55.162874 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75b02f-998b-4479-bef4-6b8b4091feed None admin] RBAC: Adding query filter params (role.id=910821ade2904d23b75fa69e040fb82d, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=6a889616f86346f48b7573969174ee57) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:55.163187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75b02f-998b-4479-bef4-6b8b4091feed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:55.165206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75b02f-998b-4479-bef4-6b8b4091feed None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:55.171744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 338/679] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:55.171744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942d4fb-9594-4e84-b956-5d12451bcb3f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:55.188242 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 342/680] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:08:54 2022] GET /identity/v3/role_assignments?user.id=6a889616f86346f48b7573969174ee57&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=910821ade2904d23b75fa69e040fb82d => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:55.189252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75b02f-998b-4479-bef4-6b8b4091feed None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:55.203722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:55.278651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942d4fb-9594-4e84-b956-5d12451bcb3f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:55.319417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387ac39-52de-47f1-a265-6aba7d1dcb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:55.320802 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387ac39-52de-47f1-a265-6aba7d1dcb18 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:55.321076 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387ac39-52de-47f1-a265-6aba7d1dcb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:55.321349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387ac39-52de-47f1-a265-6aba7d1dcb18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:55.332697 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387ac39-52de-47f1-a265-6aba7d1dcb18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:55.334463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387ac39-52de-47f1-a265-6aba7d1dcb18 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:55.352678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-2387ac39-52de-47f1-a265-6aba7d1dcb18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:55.352678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:55.352678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:55.352678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:55.352678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:55.352678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:55.352678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:55.352678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:55.352678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:55.352678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:55.352678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:55.352678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:55.352678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:55.352678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:55.352678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:55.352678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:55.353696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:55.354813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:55.354813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:55.354813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:55.354813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 339/681] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:55.361802 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387ac39-52de-47f1-a265-6aba7d1dcb18 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:55.472901 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:55.489516 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75b02f-998b-4479-bef4-6b8b4091feed None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:55.491836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387ac39-52de-47f1-a265-6aba7d1dcb18 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:55.543172 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c32310-8fa6-4cae-80cf-1cf3df1cb7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:55.543911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c32310-8fa6-4cae-80cf-1cf3df1cb7bf None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:55.544194 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c32310-8fa6-4cae-80cf-1cf3df1cb7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:55.544468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c32310-8fa6-4cae-80cf-1cf3df1cb7bf None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:55.545207 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c32310-8fa6-4cae-80cf-1cf3df1cb7bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:55.545629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c32310-8fa6-4cae-80cf-1cf3df1cb7bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:55.546574 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274549a7-b41c-481f-b586-a65597e71e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:55.547012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c32310-8fa6-4cae-80cf-1cf3df1cb7bf None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:55.547176 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274549a7-b41c-481f-b586-a65597e71e13 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:55.547460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274549a7-b41c-481f-b586-a65597e71e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:55.547772 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274549a7-b41c-481f-b586-a65597e71e13 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:55.548449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274549a7-b41c-481f-b586-a65597e71e13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:55.548749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274549a7-b41c-481f-b586-a65597e71e13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:55.550160 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274549a7-b41c-481f-b586-a65597e71e13 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:55.564057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 340/682] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:55.564884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c32310-8fa6-4cae-80cf-1cf3df1cb7bf None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:55.566510 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 343/683] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:08:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:55.567578 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274549a7-b41c-481f-b586-a65597e71e13 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:55.597326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:55.688754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c32310-8fa6-4cae-80cf-1cf3df1cb7bf None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:55.689196 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274549a7-b41c-481f-b586-a65597e71e13 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:55.724523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d9ed6-416a-473a-91e0-a5325c2e0196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:55.725184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d9ed6-416a-473a-91e0-a5325c2e0196 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:55.725469 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d9ed6-416a-473a-91e0-a5325c2e0196 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:55.725743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d9ed6-416a-473a-91e0-a5325c2e0196 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:55.731342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad69fb-f2da-4bba-949d-2e5968cc1a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:55.731997 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad69fb-f2da-4bba-949d-2e5968cc1a7a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:55.732278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad69fb-f2da-4bba-949d-2e5968cc1a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:55.732553 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad69fb-f2da-4bba-949d-2e5968cc1a7a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:55.736782 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d9ed6-416a-473a-91e0-a5325c2e0196 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:08:55.740743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad69fb-f2da-4bba-949d-2e5968cc1a7a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:55.742277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-5f4d9ed6-416a-473a-91e0-a5325c2e0196 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:08:55.742277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:55.742277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:55.742277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:55.742277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:55.742277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:55.742277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:55.742277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:55.742277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:55.742277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:55.742277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:55.742277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:55.742277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:08:55.742277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:08:55.742277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:08:55.742277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:08:55.743208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:08:55.743208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:08:55.743208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:08:55.743208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:08:55.743208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:55.743208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:55.743208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:55.743208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:55.743208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:55.743208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:55.743208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:55.743208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:55.743208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:55.743208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:55.743208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:55.743208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:55.744535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:55.744535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:55.744535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:55.744535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:55.744535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:08:55.744535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:08:55.744535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:08:55.744535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:08:55.744535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:08:55.744535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:08:55.744535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:08:55.744535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:55.745205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad69fb-f2da-4bba-949d-2e5968cc1a7a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:55.745286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 341/684] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:08:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:55.745286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d9ed6-416a-473a-91e0-a5325c2e0196 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:55.753975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:55.756284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-9cad69fb-f2da-4bba-949d-2e5968cc1a7a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 12:08:55.756284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:55.756284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:55.756284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:55.756284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:55.756284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:55.756284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:55.756284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:55.756284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:55.756284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:55.756284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:55.756284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:55.756284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:08:55.756284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:08:55.756284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:08:55.756284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:55.758584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:55.760960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:55.760960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:55.760960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:08:55.760960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:08:55.760960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:08:55.760960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:08:55.760960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:08:55.760960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:08:55.760960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 12:08:55.760960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:55.760960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 344/685] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:08:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:55.760960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad69fb-f2da-4bba-949d-2e5968cc1a7a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:55.850745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d9ed6-416a-473a-91e0-a5325c2e0196 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:55.864644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:55.888030 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4068f-af3b-4604-9d73-976bdebbe31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:55.888789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4068f-af3b-4604-9d73-976bdebbe31f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:55.889194 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4068f-af3b-4604-9d73-976bdebbe31f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:55.889572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4068f-af3b-4604-9d73-976bdebbe31f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:55.890394 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4068f-af3b-4604-9d73-976bdebbe31f None admin] RBAC: Adding query filter params (role.id=910821ade2904d23b75fa69e040fb82d, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=2e71f9c1626f4f58916a5dd5c5987ccb) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:55.890710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad69fb-f2da-4bba-949d-2e5968cc1a7a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:55.890889 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4068f-af3b-4604-9d73-976bdebbe31f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:55.893135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4068f-af3b-4604-9d73-976bdebbe31f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:55.907547 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 342/686] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:08:55 2022] GET /identity/v3/role_assignments?user.id=2e71f9c1626f4f58916a5dd5c5987ccb&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=910821ade2904d23b75fa69e040fb82d => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:55.907547 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4068f-af3b-4604-9d73-976bdebbe31f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:55.950327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3598a3-1413-4649-9290-0d1f5e9470ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:55.950865 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3598a3-1413-4649-9290-0d1f5e9470ec None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:55.951155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3598a3-1413-4649-9290-0d1f5e9470ec None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:55.951737 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3598a3-1413-4649-9290-0d1f5e9470ec None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:55.952424 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3598a3-1413-4649-9290-0d1f5e9470ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:55.952749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3598a3-1413-4649-9290-0d1f5e9470ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:55.954132 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3598a3-1413-4649-9290-0d1f5e9470ec None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:55.969361 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 345/687] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:08:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:55.980358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3598a3-1413-4649-9290-0d1f5e9470ec None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:56.050103 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:56.105088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4068f-af3b-4604-9d73-976bdebbe31f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:56.166223 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49d37a-1252-422e-875d-fa4c357329ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:56.167110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49d37a-1252-422e-875d-fa4c357329ba None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:56.167389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49d37a-1252-422e-875d-fa4c357329ba None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:56.167693 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49d37a-1252-422e-875d-fa4c357329ba None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:56.168425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49d37a-1252-422e-875d-fa4c357329ba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:56.168534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3598a3-1413-4649-9290-0d1f5e9470ec None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:56.168728 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49d37a-1252-422e-875d-fa4c357329ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:56.170434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49d37a-1252-422e-875d-fa4c357329ba None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:56.193330 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 343/688] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Oct 6 12:08:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:56.199774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49d37a-1252-422e-875d-fa4c357329ba None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:56.210393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bdcc6-122d-4d91-a777-9fcb24fa0d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:56.211489 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bdcc6-122d-4d91-a777-9fcb24fa0d29 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:56.211810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bdcc6-122d-4d91-a777-9fcb24fa0d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:56.212082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bdcc6-122d-4d91-a777-9fcb24fa0d29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:56.224753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bdcc6-122d-4d91-a777-9fcb24fa0d29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:56.226859 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bdcc6-122d-4d91-a777-9fcb24fa0d29 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:56.231842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:56.257602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-e64bdcc6-122d-4d91-a777-9fcb24fa0d29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:56.257602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:56.257602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:56.257602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:56.257602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:56.257602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:56.257602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:56.257602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:56.257602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:56.257602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:56.257602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:56.257602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:56.257602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:56.257602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:56.257602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:56.257602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:56.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:56.259611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:56.259611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:56.259611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:56.261217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 346/689] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:56.266881 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bdcc6-122d-4d91-a777-9fcb24fa0d29 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:56.344766 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49d37a-1252-422e-875d-fa4c357329ba None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:56.377430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:56.387260 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc942c-6521-4c91-b38d-c8c1872a00ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:56.387915 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc942c-6521-4c91-b38d-c8c1872a00ed None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:56.388225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc942c-6521-4c91-b38d-c8c1872a00ed None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:56.388455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc942c-6521-4c91-b38d-c8c1872a00ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:56.400085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bdcc6-122d-4d91-a777-9fcb24fa0d29 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:56.438144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415659b2-36b3-479a-8b9a-00752b9a679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:56.439151 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415659b2-36b3-479a-8b9a-00752b9a679b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:56.439687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415659b2-36b3-479a-8b9a-00752b9a679b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:56.440245 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415659b2-36b3-479a-8b9a-00752b9a679b None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:56.441186 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415659b2-36b3-479a-8b9a-00752b9a679b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:56.441727 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415659b2-36b3-479a-8b9a-00752b9a679b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:56.443321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415659b2-36b3-479a-8b9a-00752b9a679b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:56.457256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 347/690] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:56.462371 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc942c-6521-4c91-b38d-c8c1872a00ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:56.462371 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc942c-6521-4c91-b38d-c8c1872a00ed None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:56.463645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415659b2-36b3-479a-8b9a-00752b9a679b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:56.525654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:56.572061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 344/691] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 12:08:56.573743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415659b2-36b3-479a-8b9a-00752b9a679b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:56.583599 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc942c-6521-4c91-b38d-c8c1872a00ed None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:56.609281 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cb126-e2e2-40e4-a48c-51270a1c3e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:56.610025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cb126-e2e2-40e4-a48c-51270a1c3e23 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:56.610399 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cb126-e2e2-40e4-a48c-51270a1c3e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:56.610746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cb126-e2e2-40e4-a48c-51270a1c3e23 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:56.625535 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8cb126-e2e2-40e4-a48c-51270a1c3e23 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:56.625535 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8cb126-e2e2-40e4-a48c-51270a1c3e23 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:56.630060 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:56.635321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-6d8cb126-e2e2-40e4-a48c-51270a1c3e23 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:08:56.635321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:56.635321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:56.635321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:56.635321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:56.635321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:56.635321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:56.635321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:56.635321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:56.635321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:56.635321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:56.635321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:56.635321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:08:56.635321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:08:56.635321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:08:56.635321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:56.636593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:56.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:56.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:56.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:08:56.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:08:56.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:08:56.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:08:56.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:08:56.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:08:56.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:08:56.637623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:56.643898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 348/692] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Oct 6 12:08:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:56.645445 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cb126-e2e2-40e4-a48c-51270a1c3e23 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:56.756663 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cb126-e2e2-40e4-a48c-51270a1c3e23 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:56.779636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:56.793666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c0d29-0612-47b6-8b01-fe603aecf04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:56.794585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c0d29-0612-47b6-8b01-fe603aecf04f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:56.795019 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c0d29-0612-47b6-8b01-fe603aecf04f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:56.795444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c0d29-0612-47b6-8b01-fe603aecf04f None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:56.796338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159c0d29-0612-47b6-8b01-fe603aecf04f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:56.796796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159c0d29-0612-47b6-8b01-fe603aecf04f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:56.798408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159c0d29-0612-47b6-8b01-fe603aecf04f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:56.817807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 349/693] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Oct 6 12:08:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:56.818749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc942c-6521-4c91-b38d-c8c1872a00ed None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:56.824065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff52f8-9b20-4aa9-b146-d7d0bd0f4ef6 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:56.824536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff52f8-9b20-4aa9-b146-d7d0bd0f4ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:56.824992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff52f8-9b20-4aa9-b146-d7d0bd0f4ef6 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:56.826725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 350/694] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:56.827652 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff52f8-9b20-4aa9-b146-d7d0bd0f4ef6 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:56.858857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6bf00-e260-4e24-8dca-5d8ded38cb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:56.859587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6bf00-e260-4e24-8dca-5d8ded38cb82 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:56.859822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6bf00-e260-4e24-8dca-5d8ded38cb82 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:56.860097 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6bf00-e260-4e24-8dca-5d8ded38cb82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:56.871049 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6bf00-e260-4e24-8dca-5d8ded38cb82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:56.872270 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6bf00-e260-4e24-8dca-5d8ded38cb82 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:56.887623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-fbd6bf00-e260-4e24-8dca-5d8ded38cb82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:56.887623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:56.887623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:56.887623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:56.887623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:56.887623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:56.887623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:56.887623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:56.887623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:56.887623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:56.887623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:56.887623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:56.887623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:56.887623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:56.887623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:56.887623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:56.888679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:56.889604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:56.889604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:56.889604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:56.893496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 345/695] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:56.894760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb717a-251c-4b23-8ead-a35f3780ac88 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:56.895048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb717a-251c-4b23-8ead-a35f3780ac88 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:56.895339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb717a-251c-4b23-8ead-a35f3780ac88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:56.914452 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-72bb717a-251c-4b23-8ead-a35f3780ac88 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:56.915708 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-72bb717a-251c-4b23-8ead-a35f3780ac88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:56.950788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff52f8-9b20-4aa9-b146-d7d0bd0f4ef6 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:56.989263 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa5de5-613a-41f1-b829-1f62185b6cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:56.990290 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa5de5-613a-41f1-b829-1f62185b6cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:56.990837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa5de5-613a-41f1-b829-1f62185b6cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:56.991315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa5de5-613a-41f1-b829-1f62185b6cd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:57.002457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 346/696] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:57.003780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb717a-251c-4b23-8ead-a35f3780ac88 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:57.018547 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:57.052837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:57.103538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa5de5-613a-41f1-b829-1f62185b6cd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:57.105327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa5de5-613a-41f1-b829-1f62185b6cd0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:57.186421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb717a-251c-4b23-8ead-a35f3780ac88 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:57.265858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bec9a-5cf8-4f00-92c3-917083e0d63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:57.267473 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bec9a-5cf8-4f00-92c3-917083e0d63b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:57.268273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bec9a-5cf8-4f00-92c3-917083e0d63b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:57.270075 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bec9a-5cf8-4f00-92c3-917083e0d63b None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:57.270927 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5bec9a-5cf8-4f00-92c3-917083e0d63b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:57.271693 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5bec9a-5cf8-4f00-92c3-917083e0d63b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:57.274548 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5bec9a-5cf8-4f00-92c3-917083e0d63b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:57.300225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 347/697] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:57.306910 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 351/698] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:08:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:08:57.308253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bec9a-5cf8-4f00-92c3-917083e0d63b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:57.323392 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa5de5-613a-41f1-b829-1f62185b6cd0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:57.411087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:57.441264 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:57.446098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bec9a-5cf8-4f00-92c3-917083e0d63b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:57.475715 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa5de5-613a-41f1-b829-1f62185b6cd0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:57.483023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8073bf-ad19-4a9b-9e03-93a878562df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:57.483821 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8073bf-ad19-4a9b-9e03-93a878562df9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:57.484197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8073bf-ad19-4a9b-9e03-93a878562df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:57.484564 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8073bf-ad19-4a9b-9e03-93a878562df9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:57.492307 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8073bf-ad19-4a9b-9e03-93a878562df9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:57.494069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8073bf-ad19-4a9b-9e03-93a878562df9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:57.505917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-ed8073bf-ad19-4a9b-9e03-93a878562df9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:57.505917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:57.505917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:57.505917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:57.505917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:57.505917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:57.505917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:57.505917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:57.505917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:57.505917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:57.505917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:57.505917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:57.505917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:08:57.505917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:08:57.505917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:57.506920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:57.507984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:08:57.507984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:08:57.507984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:08:57.507984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:08:57.507984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:08:57.507984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:08:57.507984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:57.507984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:57.507984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 348/699] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:57.509189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8073bf-ad19-4a9b-9e03-93a878562df9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:57.516505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8eda2-e7e2-4312-a09d-34b1847d12fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:57.517149 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8eda2-e7e2-4312-a09d-34b1847d12fa None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:57.517430 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8eda2-e7e2-4312-a09d-34b1847d12fa None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:57.517701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8eda2-e7e2-4312-a09d-34b1847d12fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:57.533091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8eda2-e7e2-4312-a09d-34b1847d12fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:57.534451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8eda2-e7e2-4312-a09d-34b1847d12fa None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:57.556902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-b9d8eda2-e7e2-4312-a09d-34b1847d12fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:57.556902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:57.556902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:57.556902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:57.556902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:57.556902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:57.556902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:57.556902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:57.556902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:57.556902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:57.556902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:57.556902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:57.556902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:57.556902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:57.556902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:57.556902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:57.557961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:57.558915 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:57.558915 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:57.558915 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:57.561916 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 352/700] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:08:57.563347 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8eda2-e7e2-4312-a09d-34b1847d12fa None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:57.624957 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8073bf-ad19-4a9b-9e03-93a878562df9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:57.655303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:57.662781 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9b9d5-22ce-41b0-869b-7c5a94bba51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:57.663971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9b9d5-22ce-41b0-869b-7c5a94bba51e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:57.664253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9b9d5-22ce-41b0-869b-7c5a94bba51e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:57.664521 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9b9d5-22ce-41b0-869b-7c5a94bba51e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:57.665164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9b9d5-22ce-41b0-869b-7c5a94bba51e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:57.666921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9b9d5-22ce-41b0-869b-7c5a94bba51e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-80c9b9d5-22ce-41b0-869b-7c5a94bba51e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:57.679600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:57.680825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:57.680825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:57.680825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:57.680825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:57.680825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:57.680825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:57.680825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:08:57.680825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:08:57.680825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:08:57.680825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:08:57.680825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:57.680825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:57.680825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:57.680825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:57.680825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:57.680825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:57.681712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:57.681712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:57.681712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:57.681712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:57.681712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:57.681712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:57.681712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:57.681712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:57.681712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:57.681712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:57.681712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:08:57.681712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:08:57.681712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 12:08:57.681712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:57.685929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 349/701] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Oct 6 12:08:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:57.687113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9b9d5-22ce-41b0-869b-7c5a94bba51e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:57.716883 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8eda2-e7e2-4312-a09d-34b1847d12fa None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:57.748163 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:57.758820 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e57ae2-2ee5-445e-8024-5714fa289e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:57.759483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e57ae2-2ee5-445e-8024-5714fa289e68 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:57.759805 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e57ae2-2ee5-445e-8024-5714fa289e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:57.760075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e57ae2-2ee5-445e-8024-5714fa289e68 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:57.760824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e57ae2-2ee5-445e-8024-5714fa289e68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:57.761153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e57ae2-2ee5-445e-8024-5714fa289e68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:57.762787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e57ae2-2ee5-445e-8024-5714fa289e68 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:57.775584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 353/702] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:08:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:57.776429 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e57ae2-2ee5-445e-8024-5714fa289e68 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:57.827982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9b9d5-22ce-41b0-869b-7c5a94bba51e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:57.861991 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee62c0-79a7-4ebc-a689-f7d96335a539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:57.862601 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee62c0-79a7-4ebc-a689-f7d96335a539 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:57.862884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee62c0-79a7-4ebc-a689-f7d96335a539 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:57.863161 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee62c0-79a7-4ebc-a689-f7d96335a539 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:57.863835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee62c0-79a7-4ebc-a689-f7d96335a539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:57.864145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee62c0-79a7-4ebc-a689-f7d96335a539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:57.865503 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee62c0-79a7-4ebc-a689-f7d96335a539 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:57.876691 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 350/703] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:57.877473 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee62c0-79a7-4ebc-a689-f7d96335a539 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:57.887861 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e57ae2-2ee5-445e-8024-5714fa289e68 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:57.925296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7c338-f8ca-4a8a-81a4-df104dac6d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:57.926092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7c338-f8ca-4a8a-81a4-df104dac6d28 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:57.926516 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7c338-f8ca-4a8a-81a4-df104dac6d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:57.926929 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7c338-f8ca-4a8a-81a4-df104dac6d28 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:57.927852 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7c338-f8ca-4a8a-81a4-df104dac6d28 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:57.928397 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7c338-f8ca-4a8a-81a4-df104dac6d28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:57.929952 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7c338-f8ca-4a8a-81a4-df104dac6d28 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:57.933563 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:57.940116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 354/704] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Oct 6 12:08:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:57.940116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee476dd-d27a-41e2-8fbf-5e07bc57a1c7 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:57.940116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee476dd-d27a-41e2-8fbf-5e07bc57a1c7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:57.940116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee476dd-d27a-41e2-8fbf-5e07bc57a1c7 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:57.940116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 355/705] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:08:57.940773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee476dd-d27a-41e2-8fbf-5e07bc57a1c7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:57.988699 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee62c0-79a7-4ebc-a689-f7d96335a539 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:58.001629 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:58.023728 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedd3ff-6d48-43e0-8b71-d460186959cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:58.024340 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedd3ff-6d48-43e0-8b71-d460186959cc None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:58.024631 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedd3ff-6d48-43e0-8b71-d460186959cc None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:58.024910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedd3ff-6d48-43e0-8b71-d460186959cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:58.025638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedd3ff-6d48-43e0-8b71-d460186959cc None admin] RBAC: Adding query filter params (role.id=910821ade2904d23b75fa69e040fb82d, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=3deb4b248f884c19bf1e444f2fe0923a) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:58.025949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedd3ff-6d48-43e0-8b71-d460186959cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:58.027393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedd3ff-6d48-43e0-8b71-d460186959cc None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:58.033807 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 351/706] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:08:57 2022] GET /identity/v3/role_assignments?user.id=3deb4b248f884c19bf1e444f2fe0923a&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=910821ade2904d23b75fa69e040fb82d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:58.034595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedd3ff-6d48-43e0-8b71-d460186959cc None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:58.070156 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee476dd-d27a-41e2-8fbf-5e07bc57a1c7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:58.113889 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd43de4-698b-43b7-8688-73d332356b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:58.114984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd43de4-698b-43b7-8688-73d332356b18 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:58.115412 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd43de4-698b-43b7-8688-73d332356b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:58.115854 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd43de4-698b-43b7-8688-73d332356b18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:58.124090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd43de4-698b-43b7-8688-73d332356b18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:58.125896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd43de4-698b-43b7-8688-73d332356b18 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:58.142774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-ddd43de4-698b-43b7-8688-73d332356b18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:58.142774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:58.142774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:58.142774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:58.142774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:58.142774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:58.142774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:58.142774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:58.142774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:58.142774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:58.142774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:58.142774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:58.142774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:08:58.142774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:08:58.142774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:58.144175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:58.145166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:08:58.145166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:08:58.145166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:08:58.145166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:08:58.145166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:08:58.145166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:08:58.145166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:08:58.145166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:08:58.148228 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 356/707] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:08:58.149822 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2198d-fa0e-4146-9324-231330a1b1da None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:58.150319 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2198d-fa0e-4146-9324-231330a1b1da None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:58.150837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2198d-fa0e-4146-9324-231330a1b1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:58.171571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f5f2198d-fa0e-4146-9324-231330a1b1da None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:58.173285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f5f2198d-fa0e-4146-9324-231330a1b1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:58.175128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedd3ff-6d48-43e0-8b71-d460186959cc None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:58.195141 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:58.219683 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf2af0-65be-458e-a54c-c26c377a6750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:58.220312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf2af0-65be-458e-a54c-c26c377a6750 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:58.220598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf2af0-65be-458e-a54c-c26c377a6750 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:58.220877 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf2af0-65be-458e-a54c-c26c377a6750 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:58.221524 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf2af0-65be-458e-a54c-c26c377a6750 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:58.221835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf2af0-65be-458e-a54c-c26c377a6750 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:58.223271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf2af0-65be-458e-a54c-c26c377a6750 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:58.231082 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 352/708] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Oct 6 12:08:58 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:58.232023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf2af0-65be-458e-a54c-c26c377a6750 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:58.278381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 357/709] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:08:58.356594 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf2af0-65be-458e-a54c-c26c377a6750 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:58.396508 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1abdda-50dc-46eb-9848-0e9c5f9cd56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:58.397111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1abdda-50dc-46eb-9848-0e9c5f9cd56f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:58.397395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1abdda-50dc-46eb-9848-0e9c5f9cd56f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:58.397772 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1abdda-50dc-46eb-9848-0e9c5f9cd56f None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:58.398521 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1abdda-50dc-46eb-9848-0e9c5f9cd56f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:58.398828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1abdda-50dc-46eb-9848-0e9c5f9cd56f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:58.400333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1abdda-50dc-46eb-9848-0e9c5f9cd56f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:58.414921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 353/710] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:08:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:58.420893 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2198d-fa0e-4146-9324-231330a1b1da None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:58.472554 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:58.553112 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2198d-fa0e-4146-9324-231330a1b1da None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:58.586548 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1abdda-50dc-46eb-9848-0e9c5f9cd56f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:58.592544 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4016aa4-40cc-42d9-9dc8-055a185f454f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:58.593162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4016aa4-40cc-42d9-9dc8-055a185f454f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:58.593447 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4016aa4-40cc-42d9-9dc8-055a185f454f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:58.593723 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4016aa4-40cc-42d9-9dc8-055a185f454f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:58.594453 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4016aa4-40cc-42d9-9dc8-055a185f454f None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=5a46e63d34564d34b6b5752706375798) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:58.594762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4016aa4-40cc-42d9-9dc8-055a185f454f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:58.596325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4016aa4-40cc-42d9-9dc8-055a185f454f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:58.602997 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 358/711] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:08:58 2022] GET /identity/v3/role_assignments?user.id=5a46e63d34564d34b6b5752706375798&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:58.638795 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:58.661130 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eeaab9-899c-41cb-abda-91ae85067e64 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:58.661315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eeaab9-899c-41cb-abda-91ae85067e64 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:58.661594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eeaab9-899c-41cb-abda-91ae85067e64 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:58.662942 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 359/712] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:08:58.669187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965240e6-3b72-4dd5-ac10-ab2453087fed None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:58.669467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965240e6-3b72-4dd5-ac10-ab2453087fed None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:58.669740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965240e6-3b72-4dd5-ac10-ab2453087fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:58.695272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-965240e6-3b72-4dd5-ac10-ab2453087fed None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:08:58.696603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-965240e6-3b72-4dd5-ac10-ab2453087fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:08:58.745860 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1abdda-50dc-46eb-9848-0e9c5f9cd56f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:58.806970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bb037-9238-4d89-b6ba-af936577b621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:58.808198 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bb037-9238-4d89-b6ba-af936577b621 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:58.808482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bb037-9238-4d89-b6ba-af936577b621 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:58.808750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bb037-9238-4d89-b6ba-af936577b621 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:58.820221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128bb037-9238-4d89-b6ba-af936577b621 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:58.821670 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128bb037-9238-4d89-b6ba-af936577b621 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:58.839556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-128bb037-9238-4d89-b6ba-af936577b621 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:58.839556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:58.839556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:58.839556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:58.839556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:58.839556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:58.839556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:58.839556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:58.839556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:58.839556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:58.839556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:58.839556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:58.839556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:08:58.839556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:08:58.839556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:08:58.839556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:58.840711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:08:58.841636 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:08:58.841636 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:08:58.841636 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:58.841636 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 354/713] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:08:58.849316 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bb037-9238-4d89-b6ba-af936577b621 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:58.856876 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 360/714] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 12:08:58.945356 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:58.952167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bb037-9238-4d89-b6ba-af936577b621 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:58.998829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4033fa-8d23-41f5-8c37-291eea9df09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:58.999450 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4033fa-8d23-41f5-8c37-291eea9df09f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:58.999759 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4033fa-8d23-41f5-8c37-291eea9df09f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:59.000090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4033fa-8d23-41f5-8c37-291eea9df09f None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:59.000737 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4033fa-8d23-41f5-8c37-291eea9df09f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:59.001092 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4033fa-8d23-41f5-8c37-291eea9df09f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:59.002470 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4033fa-8d23-41f5-8c37-291eea9df09f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:59.020258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 355/715] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:08:59.026187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965240e6-3b72-4dd5-ac10-ab2453087fed None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:59.079851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:59.156315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965240e6-3b72-4dd5-ac10-ab2453087fed None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:59.197055 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db47016-d951-45d9-8f14-1de0d071e509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:59.197601 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db47016-d951-45d9-8f14-1de0d071e509 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:59.197893 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db47016-d951-45d9-8f14-1de0d071e509 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:59.198171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db47016-d951-45d9-8f14-1de0d071e509 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:59.198890 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db47016-d951-45d9-8f14-1de0d071e509 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:59.200352 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db47016-d951-45d9-8f14-1de0d071e509 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:59.232710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 361/716] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:08:59 2022] POST /identity/v3/users => generated 254 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:08:59.232924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4033fa-8d23-41f5-8c37-291eea9df09f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:59.374221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4033fa-8d23-41f5-8c37-291eea9df09f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:59.425131 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3385d2b-e798-4ab3-b436-af80358a6013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:59.425791 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3385d2b-e798-4ab3-b436-af80358a6013 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:59.426075 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3385d2b-e798-4ab3-b436-af80358a6013 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:59.426354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3385d2b-e798-4ab3-b436-af80358a6013 None admin] PATH_INFO: `/v3/services/network` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:59.426945 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3385d2b-e798-4ab3-b436-af80358a6013 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:59.428353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3385d2b-e798-4ab3-b436-af80358a6013 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-c3385d2b-e798-4ab3-b436-af80358a6013 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:08:59.443856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:08:59.445032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:08:59.445032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:08:59.445032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:08:59.445032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:08:59.445032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:08:59.445032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:08:59.445032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:08:59.445032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:08:59.445032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:08:59.445032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:08:59.445032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:08:59.445032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:08:59.445032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:08:59.445032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:08:59.445032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:08:59.445032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:08:59.445910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:08:59.445910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:08:59.445910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:08:59.445910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:08:59.445910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:08:59.445910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:08:59.445910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:08:59.445910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:08:59.445910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:08:59.445910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:08:59.445910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:08:59.445910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:08:59.445910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 12:08:59.445910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:08:59.445910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 356/717] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:08:59 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:08:59.451001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db47016-d951-45d9-8f14-1de0d071e509 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:59.559953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:59.568390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db47016-d951-45d9-8f14-1de0d071e509 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:59.605766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d860465-9b90-42de-b8fd-7d21e64866a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:59.606386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d860465-9b90-42de-b8fd-7d21e64866a8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:59.606666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d860465-9b90-42de-b8fd-7d21e64866a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:59.606938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d860465-9b90-42de-b8fd-7d21e64866a8 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:59.607674 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d860465-9b90-42de-b8fd-7d21e64866a8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:59.607981 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d860465-9b90-42de-b8fd-7d21e64866a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:59.609357 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d860465-9b90-42de-b8fd-7d21e64866a8 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:59.614415 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 362/718] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Oct 6 12:08:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:08:59.620284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3385d2b-e798-4ab3-b436-af80358a6013 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:08:59.652619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:08:59.746143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3385d2b-e798-4ab3-b436-af80358a6013 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:08:59.781538 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee9f24-bf1b-47bc-bd15-66046a4256dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:08:59.782156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee9f24-bf1b-47bc-bd15-66046a4256dd None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:08:59.782439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee9f24-bf1b-47bc-bd15-66046a4256dd None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:08:59.782715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee9f24-bf1b-47bc-bd15-66046a4256dd None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:08:59.783345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee9f24-bf1b-47bc-bd15-66046a4256dd None admin] RBAC: Adding query filter params (type=network) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:08:59.783689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee9f24-bf1b-47bc-bd15-66046a4256dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:08:59.785066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee9f24-bf1b-47bc-bd15-66046a4256dd None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:08:59.790048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 357/719] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Oct 6 12:08:59 2022] GET /identity/v3/services?type=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:00.887153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d91d12-4694-44d7-95ff-94dc666add01 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:00.887743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d91d12-4694-44d7-95ff-94dc666add01 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:00.887743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d91d12-4694-44d7-95ff-94dc666add01 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:00.889404 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 363/720] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:00.894792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79ac59-f198-44b1-9213-ebf6d28853bb None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:00.895211 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79ac59-f198-44b1-9213-ebf6d28853bb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:00.895419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79ac59-f198-44b1-9213-ebf6d28853bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:00.918525 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ed79ac59-f198-44b1-9213-ebf6d28853bb None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:00.919748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ed79ac59-f198-44b1-9213-ebf6d28853bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:00.958987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806014a-cfab-4565-b7f3-22e0305efc76 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:00.959181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806014a-cfab-4565-b7f3-22e0305efc76 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:00.959451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806014a-cfab-4565-b7f3-22e0305efc76 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:00.961600 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 364/721] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:00.971338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294d1f2-58a1-4e5f-89ae-23fc1e4f3567 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:00.971650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294d1f2-58a1-4e5f-89ae-23fc1e4f3567 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:00.971920 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294d1f2-58a1-4e5f-89ae-23fc1e4f3567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:01.012468 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f294d1f2-58a1-4e5f-89ae-23fc1e4f3567 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:01.013162 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 358/722] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:09:01.014300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f294d1f2-58a1-4e5f-89ae-23fc1e4f3567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:01.116147 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 365/723] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:09:01.300840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c853de4-17a0-4a9e-a5cd-9b3904c9b73c None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:01.301179 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c853de4-17a0-4a9e-a5cd-9b3904c9b73c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:01.301388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c853de4-17a0-4a9e-a5cd-9b3904c9b73c None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:01.302959 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 359/724] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:01.308829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acda17f-840b-4881-a072-398f451e256c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:01.309113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acda17f-840b-4881-a072-398f451e256c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:01.309382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acda17f-840b-4881-a072-398f451e256c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:01.311328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c853de4-17a0-4a9e-a5cd-9b3904c9b73c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:01.330766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-5acda17f-840b-4881-a072-398f451e256c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:01.332643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5acda17f-840b-4881-a072-398f451e256c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:01.336370 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:01.464158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 366/725] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:09:01.496956 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c853de4-17a0-4a9e-a5cd-9b3904c9b73c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:01.529486 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acda17f-840b-4881-a072-398f451e256c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:01.554826 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6d313-cbf9-4f03-9f87-d0e85b76b79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:01.555467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6d313-cbf9-4f03-9f87-d0e85b76b79c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:01.555770 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6d313-cbf9-4f03-9f87-d0e85b76b79c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:01.555893 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:01.556070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6d313-cbf9-4f03-9f87-d0e85b76b79c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:01.575429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6d313-cbf9-4f03-9f87-d0e85b76b79c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:01.580981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-a7a6d313-cbf9-4f03-9f87-d0e85b76b79c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:09:01.580981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:01.580981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:01.580981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:01.580981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:01.580981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:01.580981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:01.580981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:01.580981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:01.580981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:01.580981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:01.580981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:01.580981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:01.580981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:01.580981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:01.580981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:01.581969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:01.581969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:01.581969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:01.581969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:01.581969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:01.581969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:01.581969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:01.581969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:01.581969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:01.581969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:01.581969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:01.581969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:01.581969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:01.581969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:01.581969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:01.581969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:01.582839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:01.582839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:01.582839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:01.582839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:01.582839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:01.582839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:01.582839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:01.582839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:01.582839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:01.582839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:01.582839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:09:01.582839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:01.582839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 360/726] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:09:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:01.596429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6d313-cbf9-4f03-9f87-d0e85b76b79c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:01.670050 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acda17f-840b-4881-a072-398f451e256c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:01.705726 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c019e-54d1-4807-9b69-9ea5a3fe1ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:01.706605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c019e-54d1-4807-9b69-9ea5a3fe1ca6 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:01.707125 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c019e-54d1-4807-9b69-9ea5a3fe1ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:01.707587 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c019e-54d1-4807-9b69-9ea5a3fe1ca6 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:01.708420 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c019e-54d1-4807-9b69-9ea5a3fe1ca6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:01.709970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c019e-54d1-4807-9b69-9ea5a3fe1ca6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:01.730112 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 367/727] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Oct 6 12:09:01 2022] POST /identity/v3/services => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:09:01.740870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:01.757609 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c019e-54d1-4807-9b69-9ea5a3fe1ca6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:01.812472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6d313-cbf9-4f03-9f87-d0e85b76b79c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:01.900992 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f3072-530d-449b-912d-5c5c5778c34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:01.902507 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f3072-530d-449b-912d-5c5c5778c34e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:01.903050 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f3072-530d-449b-912d-5c5c5778c34e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:01.903815 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f3072-530d-449b-912d-5c5c5778c34e None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:01.904709 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f3072-530d-449b-912d-5c5c5778c34e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:01.906525 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f3072-530d-449b-912d-5c5c5778c34e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:01.909998 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f3072-530d-449b-912d-5c5c5778c34e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:01.919596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:01.927725 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 361/728] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:09:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:01.930530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f3072-530d-449b-912d-5c5c5778c34e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:01.965745 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c019e-54d1-4807-9b69-9ea5a3fe1ca6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:02.003121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbbbc0-bfee-4d77-af21-01c28949c627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:02.004190 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbbbc0-bfee-4d77-af21-01c28949c627 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:02.004845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbbbc0-bfee-4d77-af21-01c28949c627 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:02.005842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbbbc0-bfee-4d77-af21-01c28949c627 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:02.021490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cbbbc0-bfee-4d77-af21-01c28949c627 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:02.029271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-70cbbbc0-bfee-4d77-af21-01c28949c627 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:09:02.029271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:02.029271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:02.029271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:02.029271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:02.029271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:02.029271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:02.029271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:02.029271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:02.029271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:02.029271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:02.029271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:02.029271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:02.029271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:02.029271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:02.029271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:02.030512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:02.030512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:02.030512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:02.030512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:02.030512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:02.030512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:02.030512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:02.030512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:02.030512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:02.030512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:02.030512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:02.030512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:02.030512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:02.030512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:02.030512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:02.030512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:02.031406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:02.031406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:02.031406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:02.031406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:02.031406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:02.031406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:02.031406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:02.031406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:02.031406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:02.031406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:02.031406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:09:02.031406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:02.034571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 368/729] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:09:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:02.038836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbbbc0-bfee-4d77-af21-01c28949c627 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:02.046207 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f3072-530d-449b-912d-5c5c5778c34e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:02.091753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:02.105401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf25c9-a2a1-4284-807b-ce96683c4c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:02.106042 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf25c9-a2a1-4284-807b-ce96683c4c67 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:02.106319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf25c9-a2a1-4284-807b-ce96683c4c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:02.106585 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf25c9-a2a1-4284-807b-ce96683c4c67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:02.124522 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf25c9-a2a1-4284-807b-ce96683c4c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:02.125970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf25c9-a2a1-4284-807b-ce96683c4c67 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:02.151446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-b0bf25c9-a2a1-4284-807b-ce96683c4c67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:02.151446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:02.151446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:02.151446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:02.151446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:02.151446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:02.151446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:02.151446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:02.151446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:02.151446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:02.151446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:02.151446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:02.151446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:02.151446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:02.151446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:02.151446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:02.152504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:02.153429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:02.153429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:02.153429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:02.156771 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 362/730] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:02.160127 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf25c9-a2a1-4284-807b-ce96683c4c67 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:02.230932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbbbc0-bfee-4d77-af21-01c28949c627 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:02.299857 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf8e1a-b4b4-4453-85eb-c58728eb1c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:02.300388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf8e1a-b4b4-4453-85eb-c58728eb1c4a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:02.300665 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf8e1a-b4b4-4453-85eb-c58728eb1c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:02.300935 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf8e1a-b4b4-4453-85eb-c58728eb1c4a None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:02.301593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf8e1a-b4b4-4453-85eb-c58728eb1c4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:02.301894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf8e1a-b4b4-4453-85eb-c58728eb1c4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:02.303254 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf8e1a-b4b4-4453-85eb-c58728eb1c4a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:02.304822 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:02.325372 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 369/731] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:09:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:02.326683 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e64639-83e0-442c-a1f5-b9a52ef9d217 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:02.326969 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e64639-83e0-442c-a1f5-b9a52ef9d217 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:02.327237 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e64639-83e0-442c-a1f5-b9a52ef9d217 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:02.328714 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 370/732] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:02.330719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e64639-83e0-442c-a1f5-b9a52ef9d217 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:02.346232 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf25c9-a2a1-4284-807b-ce96683c4c67 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:02.382333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a2a9d-2505-4c80-8e4f-86401a9fa266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:02.383403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a2a9d-2505-4c80-8e4f-86401a9fa266 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:02.383918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a2a9d-2505-4c80-8e4f-86401a9fa266 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:02.384398 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a2a9d-2505-4c80-8e4f-86401a9fa266 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:02.385383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a2a9d-2505-4c80-8e4f-86401a9fa266 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:02.385892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a2a9d-2505-4c80-8e4f-86401a9fa266 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:02.387611 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a2a9d-2505-4c80-8e4f-86401a9fa266 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:02.401376 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 363/733] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:02.407576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bff61-1432-4371-b797-632f99811e84 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:02.408170 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bff61-1432-4371-b797-632f99811e84 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:02.408654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bff61-1432-4371-b797-632f99811e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:02.428458 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-975bff61-1432-4371-b797-632f99811e84 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:02.430111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-975bff61-1432-4371-b797-632f99811e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:02.448045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e64639-83e0-442c-a1f5-b9a52ef9d217 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:02.473065 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:02.493686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cdaa6b-4421-429f-827e-a4137fc09fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:02.494362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdaa6b-4421-429f-827e-a4137fc09fee None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:02.494641 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdaa6b-4421-429f-827e-a4137fc09fee None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:02.494973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdaa6b-4421-429f-827e-a4137fc09fee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:02.509640 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cdaa6b-4421-429f-827e-a4137fc09fee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:02.511336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cdaa6b-4421-429f-827e-a4137fc09fee None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:02.514436 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 364/734] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:09:02.515580 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bff61-1432-4371-b797-632f99811e84 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:02.539618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-38cdaa6b-4421-429f-827e-a4137fc09fee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:02.539618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:02.539618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:02.539618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:02.539618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:02.539618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:02.539618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:02.539618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:02.539618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:02.539618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:02.539618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:02.539618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:02.539618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:02.539618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:02.539618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:02.539618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:02.540654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:02.541614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:02.541614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:02.541614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:02.546264 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 371/735] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:02.549696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cdaa6b-4421-429f-827e-a4137fc09fee None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:02.612845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:02.660255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bff61-1432-4371-b797-632f99811e84 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:02.669794 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cdaa6b-4421-429f-827e-a4137fc09fee None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:02.706638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32a9b6-a017-48e5-a6c9-fe410de00200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:02.707492 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146f0d3-e7c7-49c9-8768-c0f80a7f052a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:02.707891 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32a9b6-a017-48e5-a6c9-fe410de00200 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:02.708129 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146f0d3-e7c7-49c9-8768-c0f80a7f052a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:02.708411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146f0d3-e7c7-49c9-8768-c0f80a7f052a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:02.708696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146f0d3-e7c7-49c9-8768-c0f80a7f052a None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:02.709167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32a9b6-a017-48e5-a6c9-fe410de00200 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:02.709364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146f0d3-e7c7-49c9-8768-c0f80a7f052a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:02.709613 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146f0d3-e7c7-49c9-8768-c0f80a7f052a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:02.710134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32a9b6-a017-48e5-a6c9-fe410de00200 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:02.711199 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146f0d3-e7c7-49c9-8768-c0f80a7f052a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:02.719221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32a9b6-a017-48e5-a6c9-fe410de00200 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:02.721588 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32a9b6-a017-48e5-a6c9-fe410de00200 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:02.728864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 372/736] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:02.730153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fc871-d13e-49a3-b218-02b6ef904d93 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:02.730438 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fc871-d13e-49a3-b218-02b6ef904d93 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:02.730715 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fc871-d13e-49a3-b218-02b6ef904d93 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:02.735036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-0b32a9b6-a017-48e5-a6c9-fe410de00200 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 12:09:02.735036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:02.735036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:02.735036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:02.735036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:02.735036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:02.735036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:02.735036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:02.735036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:02.735036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:02.735036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:02.735036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:02.735036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:09:02.735036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:09:02.735036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:09:02.735036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:02.736302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:02.737293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:02.737293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:02.737293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:09:02.737293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:09:02.737293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:09:02.737293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:09:02.737293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:09:02.737293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:09:02.737293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 12:09:02.737293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:02.737843 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 373/737] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:02.737843 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fc871-d13e-49a3-b218-02b6ef904d93 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:02.740166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 365/738] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:09:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:02.746187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072a035-7bf7-4e29-8e5f-7af4c459d06c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:02.746187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072a035-7bf7-4e29-8e5f-7af4c459d06c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:02.746187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072a035-7bf7-4e29-8e5f-7af4c459d06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:02.763069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-0072a035-7bf7-4e29-8e5f-7af4c459d06c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:02.764273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0072a035-7bf7-4e29-8e5f-7af4c459d06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:02.796731 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:02.843342 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 366/739] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 12:09:02.844379 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072a035-7bf7-4e29-8e5f-7af4c459d06c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:02.877967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fc871-d13e-49a3-b218-02b6ef904d93 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:02.889862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:02.921943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369c901-19e3-413f-8d4c-9cc40cec61ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:02.922716 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369c901-19e3-413f-8d4c-9cc40cec61ff None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:02.923107 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369c901-19e3-413f-8d4c-9cc40cec61ff None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:02.923486 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369c901-19e3-413f-8d4c-9cc40cec61ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:02.932334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369c901-19e3-413f-8d4c-9cc40cec61ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:02.933909 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369c901-19e3-413f-8d4c-9cc40cec61ff None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:02.948179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-a369c901-19e3-413f-8d4c-9cc40cec61ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:09:02.948179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:02.948179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:02.948179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:02.948179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:02.948179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:02.948179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:02.948179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:02.948179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:02.948179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:02.948179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:02.948179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:02.948179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:09:02.948179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:09:02.948179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:09:02.948179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:02.949341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:02.950476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:02.950476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:02.950476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:09:02.950476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:09:02.950476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:09:02.950476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:09:02.950476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:09:02.950476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:09:02.950476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:09:02.950476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:02.952298 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 374/740] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Oct 6 12:09:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:02.953012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369c901-19e3-413f-8d4c-9cc40cec61ff None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:02.971986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072a035-7bf7-4e29-8e5f-7af4c459d06c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:03.021777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb5bff-0ec1-4618-8122-b78f758a0b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:03.023367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb5bff-0ec1-4618-8122-b78f758a0b14 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:03.023839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb5bff-0ec1-4618-8122-b78f758a0b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:03.024266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb5bff-0ec1-4618-8122-b78f758a0b14 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:03.025756 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb5bff-0ec1-4618-8122-b78f758a0b14 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:03.026208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb5bff-0ec1-4618-8122-b78f758a0b14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:03.028110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb5bff-0ec1-4618-8122-b78f758a0b14 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:03.052447 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 367/741] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Oct 6 12:09:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:03.053475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb5bff-0ec1-4618-8122-b78f758a0b14 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:03.075443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369c901-19e3-413f-8d4c-9cc40cec61ff None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:03.081396 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:03.116923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c98cf-73ba-4a20-a1d9-0338bc11f2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:03.117729 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c98cf-73ba-4a20-a1d9-0338bc11f2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:03.118525 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c98cf-73ba-4a20-a1d9-0338bc11f2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:03.118921 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c98cf-73ba-4a20-a1d9-0338bc11f2b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:03.130077 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c98cf-73ba-4a20-a1d9-0338bc11f2b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:03.135356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-3a7c98cf-73ba-4a20-a1d9-0338bc11f2b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:09:03.135356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:03.135356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:03.135356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:03.135356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:03.135356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:03.135356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:03.135356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:03.135356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:03.135356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:03.135356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:03.135356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:03.135356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:03.135356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:03.135356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:03.135356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:03.136475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:03.136475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:03.136475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:03.136475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:03.136475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:03.136475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:03.136475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:03.136475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:03.136475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:03.136475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:03.136475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:03.136475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:03.136475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:03.136475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:03.136475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:03.136475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:03.142226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:03.142226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:03.142226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:03.142226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:03.142226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:03.142226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:03.142226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:03.142226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:03.142226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:03.142226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:03.142226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:09:03.142226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:03.142226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 375/742] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:09:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:03.142226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c98cf-73ba-4a20-a1d9-0338bc11f2b9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:03.168382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:03.182234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb5bff-0ec1-4618-8122-b78f758a0b14 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:03.219360 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf99c9-6f87-4934-9b51-051dbacf6dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:03.220018 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf99c9-6f87-4934-9b51-051dbacf6dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:03.220299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf99c9-6f87-4934-9b51-051dbacf6dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:03.220573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf99c9-6f87-4934-9b51-051dbacf6dc6 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:03.221302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf99c9-6f87-4934-9b51-051dbacf6dc6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:03.221606 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf99c9-6f87-4934-9b51-051dbacf6dc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:03.223041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf99c9-6f87-4934-9b51-051dbacf6dc6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:03.247052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 368/743] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Oct 6 12:09:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:03.247052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf99c9-6f87-4934-9b51-051dbacf6dc6 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:03.259446 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c98cf-73ba-4a20-a1d9-0338bc11f2b9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:03.301891 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d79495-c16e-437f-868d-62aa695ac739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:03.302501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d79495-c16e-437f-868d-62aa695ac739 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:03.302783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d79495-c16e-437f-868d-62aa695ac739 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:03.303056 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d79495-c16e-437f-868d-62aa695ac739 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:03.329430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:03.351633 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf99c9-6f87-4934-9b51-051dbacf6dc6 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:03.387567 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d79495-c16e-437f-868d-62aa695ac739 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:03.388882 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd48f9-c524-4a19-a6f9-5d9607f69bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:03.389100 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d79495-c16e-437f-868d-62aa695ac739 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:03.389486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd48f9-c524-4a19-a6f9-5d9607f69bd3 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:03.389764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd48f9-c524-4a19-a6f9-5d9607f69bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:03.390032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd48f9-c524-4a19-a6f9-5d9607f69bd3 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:03.390773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd48f9-c524-4a19-a6f9-5d9607f69bd3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:03.392040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd48f9-c524-4a19-a6f9-5d9607f69bd3 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:03.417345 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:03.423877 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 369/744] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Oct 6 12:09:03 2022] POST /identity/v3/services => generated 240 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:09:03.424803 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd48f9-c524-4a19-a6f9-5d9607f69bd3 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:03.516625 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 376/745] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:09:03 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 12:09:03.517870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d79495-c16e-437f-868d-62aa695ac739 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:03.651744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d79495-c16e-437f-868d-62aa695ac739 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:03.684695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:03.688098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd48f9-c524-4a19-a6f9-5d9607f69bd3 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:03.696057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50b0a5-abdf-4a42-8c24-05bb2a8542c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:03.696693 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50b0a5-abdf-4a42-8c24-05bb2a8542c5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:03.696975 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50b0a5-abdf-4a42-8c24-05bb2a8542c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:03.697248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50b0a5-abdf-4a42-8c24-05bb2a8542c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:03.710728 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50b0a5-abdf-4a42-8c24-05bb2a8542c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:03.712993 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50b0a5-abdf-4a42-8c24-05bb2a8542c5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:03.731617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5b6a3-4cd5-4008-b6ef-33b115e65779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:03.731811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-6d50b0a5-abdf-4a42-8c24-05bb2a8542c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:03.731811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:03.731811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:03.731811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:03.731811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:03.731811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:03.731811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:03.731811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:03.731811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:03.731811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:03.731811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:03.731811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:03.731811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:03.731811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:03.731811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:03.731811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:03.732712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:03.733602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:03.733602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:03.733602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:03.733602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 377/746] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:03.734529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50b0a5-abdf-4a42-8c24-05bb2a8542c5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:03.735338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5b6a3-4cd5-4008-b6ef-33b115e65779 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:03.735852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5b6a3-4cd5-4008-b6ef-33b115e65779 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:03.736368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5b6a3-4cd5-4008-b6ef-33b115e65779 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:03.737226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5b6a3-4cd5-4008-b6ef-33b115e65779 None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:03.737976 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5b6a3-4cd5-4008-b6ef-33b115e65779 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:03.740538 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5b6a3-4cd5-4008-b6ef-33b115e65779 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:03.745339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:03.756091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 370/747] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:09:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:03.757956 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5b6a3-4cd5-4008-b6ef-33b115e65779 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:03.849556 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50b0a5-abdf-4a42-8c24-05bb2a8542c5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:03.871908 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5b6a3-4cd5-4008-b6ef-33b115e65779 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:03.885365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd5040-0910-41fd-b031-e02441aa7753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:03.885992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd5040-0910-41fd-b031-e02441aa7753 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:03.886276 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd5040-0910-41fd-b031-e02441aa7753 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:03.886546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd5040-0910-41fd-b031-e02441aa7753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:03.897807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd5040-0910-41fd-b031-e02441aa7753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:03.899256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd5040-0910-41fd-b031-e02441aa7753 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:03.909623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7004290-c760-48e4-b44e-6e4850a4d441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:03.910324 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7004290-c760-48e4-b44e-6e4850a4d441 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:03.910715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7004290-c760-48e4-b44e-6e4850a4d441 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:03.911097 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7004290-c760-48e4-b44e-6e4850a4d441 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:03.912047 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7004290-c760-48e4-b44e-6e4850a4d441 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:03.912461 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7004290-c760-48e4-b44e-6e4850a4d441 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:03.914456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7004290-c760-48e4-b44e-6e4850a4d441 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:03.915308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-90dd5040-0910-41fd-b031-e02441aa7753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:03.915308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:03.915308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:03.915308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:03.915308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:03.915308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:03.915308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:03.915308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:03.915308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:03.915308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:03.915308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:03.915308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:03.915308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:03.915308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:03.915308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:03.915308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:03.916411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:03.917497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:03.917497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:03.917497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:03.921061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 378/748] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:03.921925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd5040-0910-41fd-b031-e02441aa7753 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:03.929642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 371/749] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:03.930647 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7004290-c760-48e4-b44e-6e4850a4d441 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:03.959232 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:03.993835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:04.039421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd5040-0910-41fd-b031-e02441aa7753 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:04.039748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7004290-c760-48e4-b44e-6e4850a4d441 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:04.075734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23960ebf-93f1-41dd-a2ad-e36d382a11d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:04.076609 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23960ebf-93f1-41dd-a2ad-e36d382a11d5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:04.076948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8195b6d-3efe-464c-9b1c-0c926c0d895d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:04.077255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23960ebf-93f1-41dd-a2ad-e36d382a11d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:04.077702 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23960ebf-93f1-41dd-a2ad-e36d382a11d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:04.077815 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195b6d-3efe-464c-9b1c-0c926c0d895d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:04.078207 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195b6d-3efe-464c-9b1c-0c926c0d895d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:04.078701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195b6d-3efe-464c-9b1c-0c926c0d895d None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:04.079543 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195b6d-3efe-464c-9b1c-0c926c0d895d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:04.079959 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195b6d-3efe-464c-9b1c-0c926c0d895d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:04.081510 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195b6d-3efe-464c-9b1c-0c926c0d895d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:04.090070 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23960ebf-93f1-41dd-a2ad-e36d382a11d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:04.091556 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23960ebf-93f1-41dd-a2ad-e36d382a11d5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:04.094368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 372/750] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:04.095530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8195b6d-3efe-464c-9b1c-0c926c0d895d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:04.106623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-23960ebf-93f1-41dd-a2ad-e36d382a11d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:04.106623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:04.106623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:04.106623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:04.106623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:04.106623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:04.106623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:04.106623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:04.106623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:04.106623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:04.106623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:04.106623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:04.106623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:04.106623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:04.106623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:04.106623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:04.107823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:04.108733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:04.108733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:04.108733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:04.108733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 379/751] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:04.109251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba29b1-9348-4d0f-89a6-0e512859fca0 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:04.109539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba29b1-9348-4d0f-89a6-0e512859fca0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:04.109812 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba29b1-9348-4d0f-89a6-0e512859fca0 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:04.114876 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 380/752] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:04.114876 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba29b1-9348-4d0f-89a6-0e512859fca0 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:04.203789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:04.216833 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8195b6d-3efe-464c-9b1c-0c926c0d895d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:04.223454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba29b1-9348-4d0f-89a6-0e512859fca0 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:04.234550 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:04.253552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef365bf-012d-4581-9906-104fbe67e52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:04.254188 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef365bf-012d-4581-9906-104fbe67e52e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:04.254464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef365bf-012d-4581-9906-104fbe67e52e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:04.254733 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef365bf-012d-4581-9906-104fbe67e52e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:04.262012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef365bf-012d-4581-9906-104fbe67e52e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:04.263463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef365bf-012d-4581-9906-104fbe67e52e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:04.266728 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f885ef-b32c-4bad-bccc-a56350db9d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:04.267471 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f885ef-b32c-4bad-bccc-a56350db9d4f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:04.267882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f885ef-b32c-4bad-bccc-a56350db9d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:04.268254 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f885ef-b32c-4bad-bccc-a56350db9d4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:04.275322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-0ef365bf-012d-4581-9906-104fbe67e52e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:04.275322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:04.275322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:04.275322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:04.275322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:04.275322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:04.275322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:04.275322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:04.275322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:04.275322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:04.275322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:04.275322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:04.275322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:09:04.275322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:09:04.275322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:04.276660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:04.277786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:09:04.277786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:09:04.277786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:09:04.277786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:09:04.277786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:09:04.277786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:09:04.277786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:04.277786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:04.278311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f885ef-b32c-4bad-bccc-a56350db9d4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:04.280091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f885ef-b32c-4bad-bccc-a56350db9d4f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:04.281496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 373/753] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:04.282197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef365bf-012d-4581-9906-104fbe67e52e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:04.292912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-70f885ef-b32c-4bad-bccc-a56350db9d4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:04.292912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:04.292912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:04.292912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:04.292912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:04.292912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:04.292912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:04.292912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:04.292912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:04.292912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:04.292912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:04.292912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:04.292912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:09:04.292912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:09:04.292912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:04.293967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:04.295089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:09:04.295089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:09:04.295089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:09:04.295089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:09:04.295089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:09:04.295089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:09:04.295089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:04.295089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:04.296016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 381/754] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:04.297199 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cae1b-7c26-4ddd-8ead-bba457c2b48f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:04.297550 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cae1b-7c26-4ddd-8ead-bba457c2b48f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:04.297901 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cae1b-7c26-4ddd-8ead-bba457c2b48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:04.316394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-df2cae1b-7c26-4ddd-8ead-bba457c2b48f None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:04.317678 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-df2cae1b-7c26-4ddd-8ead-bba457c2b48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:04.388798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 382/755] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:04.390059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cae1b-7c26-4ddd-8ead-bba457c2b48f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:04.408862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef365bf-012d-4581-9906-104fbe67e52e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:04.444746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbdf33-cf06-4115-a799-6ae5d9d91bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:04.445531 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbdf33-cf06-4115-a799-6ae5d9d91bfb None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:04.445879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbdf33-cf06-4115-a799-6ae5d9d91bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:04.446244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbdf33-cf06-4115-a799-6ae5d9d91bfb None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:04.447024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbdf33-cf06-4115-a799-6ae5d9d91bfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:04.447537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbdf33-cf06-4115-a799-6ae5d9d91bfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:04.448906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbdf33-cf06-4115-a799-6ae5d9d91bfb None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:04.461970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 374/756] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:04.462754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbdf33-cf06-4115-a799-6ae5d9d91bfb None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:04.469504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:04.495541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cae1b-7c26-4ddd-8ead-bba457c2b48f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:04.500362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:04.531600 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fefa3-62f0-4407-8278-3d2c4a43cf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:04.532443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fefa3-62f0-4407-8278-3d2c4a43cf70 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:04.532885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fefa3-62f0-4407-8278-3d2c4a43cf70 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:04.533302 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fefa3-62f0-4407-8278-3d2c4a43cf70 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:04.534203 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966fefa3-62f0-4407-8278-3d2c4a43cf70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:04.534726 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966fefa3-62f0-4407-8278-3d2c4a43cf70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:04.536409 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966fefa3-62f0-4407-8278-3d2c4a43cf70 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:04.548237 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 383/757] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:09:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:04.549368 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fefa3-62f0-4407-8278-3d2c4a43cf70 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:04.582520 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbdf33-cf06-4115-a799-6ae5d9d91bfb None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:04.619966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de435997-1a4a-45e7-8673-af4dfa670fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:04.620600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de435997-1a4a-45e7-8673-af4dfa670fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:04.620876 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de435997-1a4a-45e7-8673-af4dfa670fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:04.621151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de435997-1a4a-45e7-8673-af4dfa670fa4 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:04.621934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de435997-1a4a-45e7-8673-af4dfa670fa4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:04.622396 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de435997-1a4a-45e7-8673-af4dfa670fa4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:04.624054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de435997-1a4a-45e7-8673-af4dfa670fa4 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:04.638189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 375/758] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:09:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:04.638189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de435997-1a4a-45e7-8673-af4dfa670fa4 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:04.664017 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fefa3-62f0-4407-8278-3d2c4a43cf70 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:04.697617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407404f-f36c-4296-8f60-cae1d6fffb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:04.698246 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407404f-f36c-4296-8f60-cae1d6fffb2a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:04.698530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407404f-f36c-4296-8f60-cae1d6fffb2a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:04.698801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407404f-f36c-4296-8f60-cae1d6fffb2a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:04.700896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:04.707206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407404f-f36c-4296-8f60-cae1d6fffb2a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:04.708857 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407404f-f36c-4296-8f60-cae1d6fffb2a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:04.720783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-0407404f-f36c-4296-8f60-cae1d6fffb2a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 12:09:04.720783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:04.720783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:04.720783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:04.720783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:04.720783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:04.720783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:04.720783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:04.720783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:04.720783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:04.720783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:04.720783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:04.720783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:09:04.720783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:09:04.720783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:09:04.720783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:04.721761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:04.727530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:04.727530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:04.727530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:09:04.727530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:09:04.727530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:09:04.727530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:09:04.727530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:09:04.727530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:09:04.727530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 12:09:04.727530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:04.727530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 384/759] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Oct 6 12:09:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:04.727530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407404f-f36c-4296-8f60-cae1d6fffb2a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:04.750920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:04.759591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de435997-1a4a-45e7-8673-af4dfa670fa4 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:04.793027 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a8836-0f53-4bda-a2de-925eaae2f4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:04.793687 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a8836-0f53-4bda-a2de-925eaae2f4b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:04.793964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a8836-0f53-4bda-a2de-925eaae2f4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:04.794238 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a8836-0f53-4bda-a2de-925eaae2f4b9 None admin] PATH_INFO: `/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/5a46e63d34564d34b6b5752706375798/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:04.796275 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a8836-0f53-4bda-a2de-925eaae2f4b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=314d3d13ba2e4a0f93f5d3f34c50a96f, user_id=5a46e63d34564d34b6b5752706375798, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:04.798201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a8836-0f53-4bda-a2de-925eaae2f4b9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:04.819811 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 376/760] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:09:04 2022] PUT /identity/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/5a46e63d34564d34b6b5752706375798/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:09:04.819811 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a8836-0f53-4bda-a2de-925eaae2f4b9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:04.849540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407404f-f36c-4296-8f60-cae1d6fffb2a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:04.889949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644f33c-79ac-4402-a68b-0c0954ed1af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:04.890718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644f33c-79ac-4402-a68b-0c0954ed1af8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:04.891005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644f33c-79ac-4402-a68b-0c0954ed1af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:04.891279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644f33c-79ac-4402-a68b-0c0954ed1af8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:04.892056 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644f33c-79ac-4402-a68b-0c0954ed1af8 None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=3deb4b248f884c19bf1e444f2fe0923a) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:04.892369 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644f33c-79ac-4402-a68b-0c0954ed1af8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:04.893923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644f33c-79ac-4402-a68b-0c0954ed1af8 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:04.904994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 385/761] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:09:04 2022] GET /identity/v3/role_assignments?user.id=3deb4b248f884c19bf1e444f2fe0923a&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:04.904994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644f33c-79ac-4402-a68b-0c0954ed1af8 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:04.996456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a8836-0f53-4bda-a2de-925eaae2f4b9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:05.024008 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:05.049429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4e285-b2a3-4924-bced-7ca3f5a6fd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:05.050093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4e285-b2a3-4924-bced-7ca3f5a6fd49 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:05.050375 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4e285-b2a3-4924-bced-7ca3f5a6fd49 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:05.050655 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4e285-b2a3-4924-bced-7ca3f5a6fd49 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:05.051255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4e285-b2a3-4924-bced-7ca3f5a6fd49 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:05.052773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4e285-b2a3-4924-bced-7ca3f5a6fd49 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-12f4e285-b2a3-4924-bced-7ca3f5a6fd49 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:05.064271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:05.065359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:05.065359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:05.065359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:05.065359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:05.065359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:05.065359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:05.065359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:05.065359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:05.065359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:05.065359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:05.065359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:05.065359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:05.065359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:05.065359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:05.065359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:05.065359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:05.066208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:05.066208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:05.066208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:05.066208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:05.066208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:05.066208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:05.066208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:05.066208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:05.066208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:05.066208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:05.066208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:05.066208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:05.066208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 12:09:05.066208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:05.066208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 377/762] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Oct 6 12:09:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:05.072292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4e285-b2a3-4924-bced-7ca3f5a6fd49 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:05.095668 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644f33c-79ac-4402-a68b-0c0954ed1af8 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:05.106072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:05.157204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5284c2e-9494-41d9-9880-812264a824cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:05.158358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5284c2e-9494-41d9-9880-812264a824cc None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:05.158916 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5284c2e-9494-41d9-9880-812264a824cc None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:05.159378 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5284c2e-9494-41d9-9880-812264a824cc None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:05.160338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5284c2e-9494-41d9-9880-812264a824cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:05.161093 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5284c2e-9494-41d9-9880-812264a824cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:05.164820 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5284c2e-9494-41d9-9880-812264a824cc None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:05.186902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 386/763] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Oct 6 12:09:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:05.191970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5284c2e-9494-41d9-9880-812264a824cc None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:05.205377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4e285-b2a3-4924-bced-7ca3f5a6fd49 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:05.243575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44edd46-267e-4cc9-9bad-8693d68fd4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:05.243773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44edd46-267e-4cc9-9bad-8693d68fd4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:05.244202 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44edd46-267e-4cc9-9bad-8693d68fd4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:05.244548 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44edd46-267e-4cc9-9bad-8693d68fd4c1 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:05.245255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44edd46-267e-4cc9-9bad-8693d68fd4c1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:05.245632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44edd46-267e-4cc9-9bad-8693d68fd4c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:05.247223 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44edd46-267e-4cc9-9bad-8693d68fd4c1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:05.252501 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 378/764] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Oct 6 12:09:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:05.258847 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44edd46-267e-4cc9-9bad-8693d68fd4c1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:05.386407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5284c2e-9494-41d9-9880-812264a824cc None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:05.421923 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:05.422120 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:05.450098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44edd46-267e-4cc9-9bad-8693d68fd4c1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:05.469808 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd3761-9fdd-460c-a888-564ac1990bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:05.470988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd3761-9fdd-460c-a888-564ac1990bba None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:05.472497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd3761-9fdd-460c-a888-564ac1990bba None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:05.473365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd3761-9fdd-460c-a888-564ac1990bba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:05.510704 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420f909-346a-4a45-a217-af344746b1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:05.511705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420f909-346a-4a45-a217-af344746b1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:05.512840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420f909-346a-4a45-a217-af344746b1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:05.514967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420f909-346a-4a45-a217-af344746b1b9 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:05.515868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420f909-346a-4a45-a217-af344746b1b9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:05.516383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420f909-346a-4a45-a217-af344746b1b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:05.520074 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420f909-346a-4a45-a217-af344746b1b9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:05.533842 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 379/765] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Oct 6 12:09:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:05.537553 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420f909-346a-4a45-a217-af344746b1b9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:05.621524 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd3761-9fdd-460c-a888-564ac1990bba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:05.623439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd3761-9fdd-460c-a888-564ac1990bba None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:05.716678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420f909-346a-4a45-a217-af344746b1b9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:05.767357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d80a9-c7e1-425e-b835-a473a1c2c227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:05.768334 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d80a9-c7e1-425e-b835-a473a1c2c227 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:05.768845 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d80a9-c7e1-425e-b835-a473a1c2c227 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:05.769879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d80a9-c7e1-425e-b835-a473a1c2c227 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:05.771077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d80a9-c7e1-425e-b835-a473a1c2c227 None admin] RBAC: Adding query filter params (interface=public, service_id=93c647b91550460f88392fb162ec53c7, region_id=RegionOne) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:05.771627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d80a9-c7e1-425e-b835-a473a1c2c227 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:05.774857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d80a9-c7e1-425e-b835-a473a1c2c227 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:05.787610 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 387/766] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:09:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:09:05.787610 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615aacd-d9f9-4f66-90c4-693d6e78d9e0 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:05.787610 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615aacd-d9f9-4f66-90c4-693d6e78d9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:05.787610 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615aacd-d9f9-4f66-90c4-693d6e78d9e0 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:05.791292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 388/767] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:05.791772 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 380/768] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Oct 6 12:09:05 2022] GET /identity/v3/endpoints?service_id=93c647b91550460f88392fb162ec53c7&interface=public®ion_id=RegionOne => generated 194 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:05.793210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bd19c-6334-47fe-af4c-e5ec1654b1ca None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:05.793720 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bd19c-6334-47fe-af4c-e5ec1654b1ca None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:05.794199 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bd19c-6334-47fe-af4c-e5ec1654b1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:05.794468 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615aacd-d9f9-4f66-90c4-693d6e78d9e0 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:05.817838 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9e0bd19c-6334-47fe-af4c-e5ec1654b1ca None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:05.818668 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:05.820507 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9e0bd19c-6334-47fe-af4c-e5ec1654b1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:05.835413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:05.905980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 381/769] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:05.907296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bd19c-6334-47fe-af4c-e5ec1654b1ca None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:05.925842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615aacd-d9f9-4f66-90c4-693d6e78d9e0 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:05.992136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64a002-d552-4c68-a345-bffa2c64204a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:05.994660 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64a002-d552-4c68-a345-bffa2c64204a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:05.994944 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64a002-d552-4c68-a345-bffa2c64204a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:05.995218 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64a002-d552-4c68-a345-bffa2c64204a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:06.017848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d64a002-d552-4c68-a345-bffa2c64204a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:06.019233 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d64a002-d552-4c68-a345-bffa2c64204a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:06.060927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-4d64a002-d552-4c68-a345-bffa2c64204a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:06.060927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:06.060927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:06.060927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:06.060927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:06.060927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:06.060927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:06.060927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:06.060927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:06.060927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:06.060927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:06.060927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:06.060927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:06.060927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:06.060927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:06.060927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:06.062377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:06.063295 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:06.063295 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:06.063295 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:06.063295 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 389/770] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:06.070598 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64a002-d552-4c68-a345-bffa2c64204a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:06.103076 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bd19c-6334-47fe-af4c-e5ec1654b1ca None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:06.169235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b31f65-ddfa-4067-8b1f-784fabe76c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:06.170186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b31f65-ddfa-4067-8b1f-784fabe76c5b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:06.171483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b31f65-ddfa-4067-8b1f-784fabe76c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:06.171984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b31f65-ddfa-4067-8b1f-784fabe76c5b None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:06.172791 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b31f65-ddfa-4067-8b1f-784fabe76c5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:06.175334 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b31f65-ddfa-4067-8b1f-784fabe76c5b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:06.187259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 382/771] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Oct 6 12:09:05 2022] GET /identity/v3/services => generated 772 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:06.211965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:06.218679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b31f65-ddfa-4067-8b1f-784fabe76c5b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:06.235947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:06.281295 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64a002-d552-4c68-a345-bffa2c64204a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:06.366714 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4dc57-c978-4e5c-be1f-f0dbc59a1405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:06.367349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4dc57-c978-4e5c-be1f-f0dbc59a1405 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:06.367670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4dc57-c978-4e5c-be1f-f0dbc59a1405 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:06.367947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4dc57-c978-4e5c-be1f-f0dbc59a1405 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:06.368698 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4dc57-c978-4e5c-be1f-f0dbc59a1405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:06.369034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4dc57-c978-4e5c-be1f-f0dbc59a1405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:06.370489 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4dc57-c978-4e5c-be1f-f0dbc59a1405 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:06.378481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b31f65-ddfa-4067-8b1f-784fabe76c5b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:06.392060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 390/772] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:06.398231 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4dc57-c978-4e5c-be1f-f0dbc59a1405 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:06.422802 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce824d-5b1e-4039-b07c-c784c667304e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:06.423448 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce824d-5b1e-4039-b07c-c784c667304e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:06.423805 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce824d-5b1e-4039-b07c-c784c667304e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:06.424085 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce824d-5b1e-4039-b07c-c784c667304e None admin] PATH_INFO: `/v3/services/network` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:06.425209 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce824d-5b1e-4039-b07c-c784c667304e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:06.426885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce824d-5b1e-4039-b07c-c784c667304e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-30ce824d-5b1e-4039-b07c-c784c667304e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:06.441869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:06.443083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:06.443083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:06.443083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:06.443083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:06.443083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:06.443083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:06.443083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:06.443083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:06.443083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:06.443083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:06.443083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:06.443083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:06.443083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:06.443083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:06.443083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:06.443083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:06.443978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:06.443978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:06.443978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:06.443978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:06.443978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:06.443978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:06.443978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:06.443978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:06.443978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:06.443978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:06.443978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:06.443978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:06.443978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 12:09:06.443978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:06.443978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 383/773] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:06 2022] GET /identity/v3/services/network => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:06.450711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce824d-5b1e-4039-b07c-c784c667304e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:06.531001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4dc57-c978-4e5c-be1f-f0dbc59a1405 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:06.545349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:06.562036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:06.571460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760604e0-3db7-4457-905d-e31ffc075751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:06.572126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760604e0-3db7-4457-905d-e31ffc075751 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:06.572409 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760604e0-3db7-4457-905d-e31ffc075751 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:06.572744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760604e0-3db7-4457-905d-e31ffc075751 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:06.582179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760604e0-3db7-4457-905d-e31ffc075751 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:06.585300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760604e0-3db7-4457-905d-e31ffc075751 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:06.590926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce824d-5b1e-4039-b07c-c784c667304e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:06.601623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-760604e0-3db7-4457-905d-e31ffc075751 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:06.601623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:06.601623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:06.601623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:06.601623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:06.601623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:06.601623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:06.601623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:06.601623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:06.601623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:06.601623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:06.601623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:06.601623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:09:06.601623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:09:06.601623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:06.602547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:06.603540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:09:06.603540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:09:06.603540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:09:06.603540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:09:06.603540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:09:06.603540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:09:06.603540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:06.603540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:06.605854 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 391/774] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:06.620725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760604e0-3db7-4457-905d-e31ffc075751 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:06.645275 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd08b9-e5d7-40ee-bca5-2c1eb1e2b5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:06.645897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd08b9-e5d7-40ee-bca5-2c1eb1e2b5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:06.646965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd08b9-e5d7-40ee-bca5-2c1eb1e2b5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:06.648749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd08b9-e5d7-40ee-bca5-2c1eb1e2b5b9 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:06.649359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcd08b9-e5d7-40ee-bca5-2c1eb1e2b5b9 None admin] RBAC: Adding query filter params (name=network) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:06.649664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcd08b9-e5d7-40ee-bca5-2c1eb1e2b5b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:06.651953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcd08b9-e5d7-40ee-bca5-2c1eb1e2b5b9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:06.661396 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 384/775] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Oct 6 12:09:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:06.667862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd08b9-e5d7-40ee-bca5-2c1eb1e2b5b9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:06.762497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760604e0-3db7-4457-905d-e31ffc075751 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:06.799498 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67acfc80-959e-4d56-95cc-64a842c79935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:06.800064 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67acfc80-959e-4d56-95cc-64a842c79935 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:06.800347 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67acfc80-959e-4d56-95cc-64a842c79935 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:06.800622 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67acfc80-959e-4d56-95cc-64a842c79935 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:06.801318 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67acfc80-959e-4d56-95cc-64a842c79935 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:06.801627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67acfc80-959e-4d56-95cc-64a842c79935 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:06.803076 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67acfc80-959e-4d56-95cc-64a842c79935 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:06.816503 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 392/776] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:09:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:06.823124 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67acfc80-959e-4d56-95cc-64a842c79935 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:06.877563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:06.893677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd08b9-e5d7-40ee-bca5-2c1eb1e2b5b9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:06.930377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35b6c2-e452-42e7-95b7-e81e4ea61824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:06.930994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35b6c2-e452-42e7-95b7-e81e4ea61824 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:06.931280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35b6c2-e452-42e7-95b7-e81e4ea61824 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:06.931573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35b6c2-e452-42e7-95b7-e81e4ea61824 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:06.932206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35b6c2-e452-42e7-95b7-e81e4ea61824 None admin] RBAC: Adding query filter params (type=network) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:06.932509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35b6c2-e452-42e7-95b7-e81e4ea61824 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:06.933876 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35b6c2-e452-42e7-95b7-e81e4ea61824 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:06.935155 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:06.937739 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67acfc80-959e-4d56-95cc-64a842c79935 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:06.939269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 385/777] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Oct 6 12:09:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:06.945080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35b6c2-e452-42e7-95b7-e81e4ea61824 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:06.978153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da769da-0317-47ee-bca2-be6a8e8a0072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:06.979144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da769da-0317-47ee-bca2-be6a8e8a0072 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:06.979710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da769da-0317-47ee-bca2-be6a8e8a0072 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:06.980244 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da769da-0317-47ee-bca2-be6a8e8a0072 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:06.981173 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da769da-0317-47ee-bca2-be6a8e8a0072 None admin] RBAC: Adding query filter params (role.id=910821ade2904d23b75fa69e040fb82d, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=96a302378dc0416d87c1cf11a3ba0a9c) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:06.981687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da769da-0317-47ee-bca2-be6a8e8a0072 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:06.983396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da769da-0317-47ee-bca2-be6a8e8a0072 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:06.990641 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 393/778] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:09:06 2022] GET /identity/v3/role_assignments?user.id=96a302378dc0416d87c1cf11a3ba0a9c&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=910821ade2904d23b75fa69e040fb82d => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:07.110484 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35b6c2-e452-42e7-95b7-e81e4ea61824 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:07.148828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c04dd9-b25f-40d4-89f3-9d2e31b2421b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:07.149455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04dd9-b25f-40d4-89f3-9d2e31b2421b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:07.149737 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04dd9-b25f-40d4-89f3-9d2e31b2421b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:07.150012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04dd9-b25f-40d4-89f3-9d2e31b2421b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:07.150723 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c04dd9-b25f-40d4-89f3-9d2e31b2421b None admin] RBAC: Adding query filter params (interface=public, service_id=f9574caf582d420b8d8204242c501db0, region_id=RegionOne) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:07.151039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c04dd9-b25f-40d4-89f3-9d2e31b2421b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:07.152467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c04dd9-b25f-40d4-89f3-9d2e31b2421b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:07.159038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 386/779] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Oct 6 12:09:06 2022] GET /identity/v3/endpoints?service_id=f9574caf582d420b8d8204242c501db0&interface=public®ion_id=RegionOne => generated 194 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:07.164713 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da769da-0317-47ee-bca2-be6a8e8a0072 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:07.286917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da769da-0317-47ee-bca2-be6a8e8a0072 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:07.311835 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:07.323866 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9dfcc-07c7-4406-9b1a-a9daae49bf21 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:07.324490 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9dfcc-07c7-4406-9b1a-a9daae49bf21 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:07.324976 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9dfcc-07c7-4406-9b1a-a9daae49bf21 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:07.326509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 387/780] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:09:07.331815 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d5e0b-ee28-4f2c-9f36-4c84577287b6 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:07.332106 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d5e0b-ee28-4f2c-9f36-4c84577287b6 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:07.332382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d5e0b-ee28-4f2c-9f36-4c84577287b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:07.341883 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd4bbc-820d-4077-b40e-b9ac7c298c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:07.342492 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd4bbc-820d-4077-b40e-b9ac7c298c37 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:07.342775 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd4bbc-820d-4077-b40e-b9ac7c298c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:07.343048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd4bbc-820d-4077-b40e-b9ac7c298c37 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:07.343688 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd4bbc-820d-4077-b40e-b9ac7c298c37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:07.345183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd4bbc-820d-4077-b40e-b9ac7c298c37 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:07.351537 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 394/781] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Oct 6 12:09:07 2022] GET /identity/v3/services => generated 772 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:07.372902 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-497d5e0b-ee28-4f2c-9f36-4c84577287b6 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:07.374052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-497d5e0b-ee28-4f2c-9f36-4c84577287b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:07.472114 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:07.493193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 388/782] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:07.674178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c056f77-4923-41ba-8fbe-a77bcaecf181 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:07.674365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c056f77-4923-41ba-8fbe-a77bcaecf181 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:07.674631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c056f77-4923-41ba-8fbe-a77bcaecf181 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:07.676349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 395/783] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:07.681463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96872965-a90a-41a4-ace2-ad34214328eb None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:07.681753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96872965-a90a-41a4-ace2-ad34214328eb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:07.682035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96872965-a90a-41a4-ace2-ad34214328eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:07.707535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-96872965-a90a-41a4-ace2-ad34214328eb None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:07.708755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-96872965-a90a-41a4-ace2-ad34214328eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:07.803093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 389/784] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:07.829231 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c056f77-4923-41ba-8fbe-a77bcaecf181 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:08.000353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c056f77-4923-41ba-8fbe-a77bcaecf181 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:08.046600 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6117ba-1761-4c02-9f62-fab27857c1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:08.047231 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6117ba-1761-4c02-9f62-fab27857c1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:08.047532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6117ba-1761-4c02-9f62-fab27857c1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:08.047807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6117ba-1761-4c02-9f62-fab27857c1d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:08.062972 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6117ba-1761-4c02-9f62-fab27857c1d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:08.067809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-0b6117ba-1761-4c02-9f62-fab27857c1d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:09:08.067809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:08.067809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:08.067809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:08.067809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:08.067809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:08.067809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:08.067809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:08.067809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:08.067809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:08.067809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:08.067809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:08.067809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:08.067809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:08.067809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:08.067809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:08.068753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:08.068753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:08.068753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:08.068753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:08.068753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:08.068753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:08.068753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:08.068753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:08.068753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:08.068753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:08.068753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:08.068753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:08.068753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:08.068753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:08.068753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:08.068753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:08.069700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:08.069700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:08.069700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:08.069700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:08.069700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:08.069700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:08.069700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:08.069700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:08.069700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:08.069700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:08.069700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:09:08.069700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:08.069700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 396/785] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:09:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:08.076668 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96872965-a90a-41a4-ace2-ad34214328eb None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:08.091829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6117ba-1761-4c02-9f62-fab27857c1d8 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:08.151883 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:08.259480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6117ba-1761-4c02-9f62-fab27857c1d8 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:08.280480 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96872965-a90a-41a4-ace2-ad34214328eb None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:08.317936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86057233-b68d-41a2-951c-f69e33a34e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:08.318628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86057233-b68d-41a2-951c-f69e33a34e27 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:08.318914 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86057233-b68d-41a2-951c-f69e33a34e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:08.319738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86057233-b68d-41a2-951c-f69e33a34e27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:08.324558 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:08.338651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14962043-7992-427f-992a-6731e73f04d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:08.339441 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14962043-7992-427f-992a-6731e73f04d1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:08.339768 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14962043-7992-427f-992a-6731e73f04d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:08.340008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14962043-7992-427f-992a-6731e73f04d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:08.340666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14962043-7992-427f-992a-6731e73f04d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:08.341218 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14962043-7992-427f-992a-6731e73f04d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:08.342572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14962043-7992-427f-992a-6731e73f04d1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:08.352448 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86057233-b68d-41a2-951c-f69e33a34e27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:08.352604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 390/786] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:09:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:08.358134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-86057233-b68d-41a2-951c-f69e33a34e27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:09:08.358134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:08.358134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:08.358134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:08.358134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:08.358134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:08.358134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:08.358134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:08.358134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:08.358134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:08.358134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:08.358134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:08.358134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:08.358134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:08.358134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:08.358134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:08.359091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:08.359091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:08.359091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:08.359091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:08.359091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:08.359091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:08.359091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:08.359091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:08.359091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:08.359091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:08.359091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:08.359091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:08.359091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:08.359091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:08.359091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:08.359091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:08.360002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:08.360002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:08.360002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:08.360002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:08.360002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:08.360002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:08.360002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:08.360002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:08.360002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:08.360002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:08.360002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:09:08.360002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:08.360002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 397/787] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:09:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:08.360716 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14962043-7992-427f-992a-6731e73f04d1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:08.368477 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86057233-b68d-41a2-951c-f69e33a34e27 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:08.558344 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14962043-7992-427f-992a-6731e73f04d1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:08.564769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:08.582504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86057233-b68d-41a2-951c-f69e33a34e27 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:08.600935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82b7f6-6308-4d4f-8474-ef3430b3658f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:08.601918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82b7f6-6308-4d4f-8474-ef3430b3658f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:08.602401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82b7f6-6308-4d4f-8474-ef3430b3658f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:08.604285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82b7f6-6308-4d4f-8474-ef3430b3658f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:08.617738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82b7f6-6308-4d4f-8474-ef3430b3658f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:08.619445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82b7f6-6308-4d4f-8474-ef3430b3658f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:08.634252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-6b82b7f6-6308-4d4f-8474-ef3430b3658f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:08.634252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:08.634252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:08.634252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:08.634252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:08.634252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:08.634252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:08.634252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:08.634252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:08.634252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:08.634252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:08.634252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:08.634252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:08.634252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:08.634252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:08.634252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:08.635251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:08.636204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:08.636204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:08.636204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:08.638239 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 391/788] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:08.644109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82b7f6-6308-4d4f-8474-ef3430b3658f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:08.649858 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a849eb-9e9f-49a0-9c0c-bede0ce56d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:08.650475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a849eb-9e9f-49a0-9c0c-bede0ce56d04 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:08.650754 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a849eb-9e9f-49a0-9c0c-bede0ce56d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:08.651024 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a849eb-9e9f-49a0-9c0c-bede0ce56d04 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:08.651713 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a849eb-9e9f-49a0-9c0c-bede0ce56d04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:08.652014 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a849eb-9e9f-49a0-9c0c-bede0ce56d04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:08.653383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a849eb-9e9f-49a0-9c0c-bede0ce56d04 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:08.671370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 398/789] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:09:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:08.678938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a849eb-9e9f-49a0-9c0c-bede0ce56d04 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:08.689832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:08.760503 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82b7f6-6308-4d4f-8474-ef3430b3658f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:08.797585 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a89626-624e-4beb-bf91-a6606ee58fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:08.798578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a89626-624e-4beb-bf91-a6606ee58fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:08.799060 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a89626-624e-4beb-bf91-a6606ee58fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:08.799571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a89626-624e-4beb-bf91-a6606ee58fc2 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:08.800404 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a89626-624e-4beb-bf91-a6606ee58fc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:08.801333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a89626-624e-4beb-bf91-a6606ee58fc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:08.802995 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a89626-624e-4beb-bf91-a6606ee58fc2 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:08.816776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 392/790] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:08.822585 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a89626-624e-4beb-bf91-a6606ee58fc2 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:08.913491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a849eb-9e9f-49a0-9c0c-bede0ce56d04 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:08.936602 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:08.943762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a89626-624e-4beb-bf91-a6606ee58fc2 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:08.952792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139de4f0-05a2-4d83-8c9f-8a64f8d1e52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:08.953746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139de4f0-05a2-4d83-8c9f-8a64f8d1e52c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:08.954200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139de4f0-05a2-4d83-8c9f-8a64f8d1e52c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:08.954625 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139de4f0-05a2-4d83-8c9f-8a64f8d1e52c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:08.966883 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139de4f0-05a2-4d83-8c9f-8a64f8d1e52c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:08.968834 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139de4f0-05a2-4d83-8c9f-8a64f8d1e52c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:08.982201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b33e7-9bc9-4242-8c5f-55de11352ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:08.982833 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b33e7-9bc9-4242-8c5f-55de11352ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:08.983300 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b33e7-9bc9-4242-8c5f-55de11352ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:08.983757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b33e7-9bc9-4242-8c5f-55de11352ed9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:08.985543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-139de4f0-05a2-4d83-8c9f-8a64f8d1e52c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:08.985543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:08.985543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:08.985543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:08.985543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:08.985543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:08.985543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:08.985543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:08.985543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:08.985543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:08.985543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:08.985543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:08.985543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:08.985543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:08.985543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:08.985543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:08.986504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:08.987439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:08.987439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:08.987439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:08.987439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 399/791] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:08.993348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f5956-7c8f-4248-a6a0-e488024e0449 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:08.993889 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f5956-7c8f-4248-a6a0-e488024e0449 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:08.994360 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f5956-7c8f-4248-a6a0-e488024e0449 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:08.994628 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b33e7-9bc9-4242-8c5f-55de11352ed9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:08.996169 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b33e7-9bc9-4242-8c5f-55de11352ed9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:08.996433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 400/792] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:08.997317 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f5956-7c8f-4248-a6a0-e488024e0449 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:09.011913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:09.015267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-460b33e7-9bc9-4242-8c5f-55de11352ed9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:09:09.015267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:09.015267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:09.015267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:09.015267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:09.015267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:09.015267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:09.015267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:09.015267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:09.015267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:09.015267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:09.015267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:09.015267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:09:09.015267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:09:09.015267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:09:09.015267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:09.016349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:09.017310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:09.017310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:09.017310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:09:09.017310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:09:09.017310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:09:09.017310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:09:09.017310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:09:09.017310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:09:09.017310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:09:09.017310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:09.019889 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 393/793] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Oct 6 12:09:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:09.024906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150eefd1-a949-4c08-b45f-b5e4f73a98f8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:09.025378 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150eefd1-a949-4c08-b45f-b5e4f73a98f8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:09.025828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150eefd1-a949-4c08-b45f-b5e4f73a98f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:09.046111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-150eefd1-a949-4c08-b45f-b5e4f73a98f8 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:09.047601 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-150eefd1-a949-4c08-b45f-b5e4f73a98f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:09.110161 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f5956-7c8f-4248-a6a0-e488024e0449 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:09.123797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 394/794] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:09.124931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150eefd1-a949-4c08-b45f-b5e4f73a98f8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:09.146523 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e5049-6e4a-42c5-92f7-4d7adf9bdb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:09.147474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e5049-6e4a-42c5-92f7-4d7adf9bdb6f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:09.148163 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e5049-6e4a-42c5-92f7-4d7adf9bdb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:09.148576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e5049-6e4a-42c5-92f7-4d7adf9bdb6f None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:09.149345 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e5049-6e4a-42c5-92f7-4d7adf9bdb6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:09.149784 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e5049-6e4a-42c5-92f7-4d7adf9bdb6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:09.151289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e5049-6e4a-42c5-92f7-4d7adf9bdb6f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:09.165151 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 401/795] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:09.170932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e5049-6e4a-42c5-92f7-4d7adf9bdb6f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:09.271899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:09.285361 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e5049-6e4a-42c5-92f7-4d7adf9bdb6f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:09.297224 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:09.325117 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150eefd1-a949-4c08-b45f-b5e4f73a98f8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:09.342555 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f99314-85b2-4070-9d6e-c1c4c6177b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:09.343412 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f99314-85b2-4070-9d6e-c1c4c6177b49 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:09.343884 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f99314-85b2-4070-9d6e-c1c4c6177b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:09.344351 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f99314-85b2-4070-9d6e-c1c4c6177b49 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:09.352827 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f99314-85b2-4070-9d6e-c1c4c6177b49 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:09.354529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f99314-85b2-4070-9d6e-c1c4c6177b49 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:09.368568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-65f99314-85b2-4070-9d6e-c1c4c6177b49 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 12:09:09.368568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:09.368568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:09.368568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:09.368568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:09.368568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:09.368568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:09.368568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:09.368568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:09.368568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:09.368568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:09.368568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:09.368568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:09:09.368568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:09:09.368568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:09:09.368568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:09.369796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:09.370988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:09.370988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:09.370988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:09:09.370988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:09:09.370988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:09:09.370988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:09:09.370988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:09:09.370988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:09:09.370988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 12:09:09.370988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:09.373530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 402/796] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:09:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:09.379715 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f99314-85b2-4070-9d6e-c1c4c6177b49 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:09.420255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7614e5-4c61-4444-96de-3218117d48a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:09.421034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7614e5-4c61-4444-96de-3218117d48a2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:09.421381 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7614e5-4c61-4444-96de-3218117d48a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:09.421722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7614e5-4c61-4444-96de-3218117d48a2 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:09.422534 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7614e5-4c61-4444-96de-3218117d48a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:09.422906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7614e5-4c61-4444-96de-3218117d48a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:09.424519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7614e5-4c61-4444-96de-3218117d48a2 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:09.466075 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 395/797] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Oct 6 12:09:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:09.472654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7614e5-4c61-4444-96de-3218117d48a2 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:09.504050 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f99314-85b2-4070-9d6e-c1c4c6177b49 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:09.534810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:09.541104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1560601-0c3b-4fa9-b965-7ec4ad996e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:09.541704 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1560601-0c3b-4fa9-b965-7ec4ad996e36 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:09.541982 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1560601-0c3b-4fa9-b965-7ec4ad996e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:09.542257 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1560601-0c3b-4fa9-b965-7ec4ad996e36 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:09.542979 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1560601-0c3b-4fa9-b965-7ec4ad996e36 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:09.543283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1560601-0c3b-4fa9-b965-7ec4ad996e36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:09.544766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1560601-0c3b-4fa9-b965-7ec4ad996e36 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:09.566234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 403/798] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Oct 6 12:09:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:09.567562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1560601-0c3b-4fa9-b965-7ec4ad996e36 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:09.602024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7614e5-4c61-4444-96de-3218117d48a2 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:09.638703 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91382f-976a-453f-bbf0-5cd63ab3602d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:09.639233 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91382f-976a-453f-bbf0-5cd63ab3602d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:09.639531 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91382f-976a-453f-bbf0-5cd63ab3602d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:09.639852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91382f-976a-453f-bbf0-5cd63ab3602d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:09.654385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:09.679550 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1560601-0c3b-4fa9-b965-7ec4ad996e36 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:09.711427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91382f-976a-453f-bbf0-5cd63ab3602d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:09.713013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91382f-976a-453f-bbf0-5cd63ab3602d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:09.713629 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558e77a-b6ec-4b01-ab10-0aeff72360d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:09.714262 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558e77a-b6ec-4b01-ab10-0aeff72360d4 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:09.714542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558e77a-b6ec-4b01-ab10-0aeff72360d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:09.714977 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558e77a-b6ec-4b01-ab10-0aeff72360d4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:09.715714 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558e77a-b6ec-4b01-ab10-0aeff72360d4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:09.717076 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558e77a-b6ec-4b01-ab10-0aeff72360d4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-e558e77a-b6ec-4b01-ab10-0aeff72360d4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:09.729590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:09.730675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:09.730675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:09.730675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:09.730675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:09.730675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:09.730675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:09.730675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:09.730675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:09.730675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:09.730675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:09.730675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:09.730675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:09.730675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:09.730675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:09.730675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:09.730675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:09.732551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:09.732551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:09.732551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:09.732551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:09.732551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:09.732551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:09.732551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:09.732551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:09.732551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:09.732551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:09.732551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:09.732551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:09.732551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 12:09:09.732551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:09.732551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 404/799] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Oct 6 12:09:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:09.733382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb596ff8-a942-46d9-b307-58c1680ec4c5 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:09.733382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb596ff8-a942-46d9-b307-58c1680ec4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:09.733382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb596ff8-a942-46d9-b307-58c1680ec4c5 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:09.737999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 405/800] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:09.738768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb596ff8-a942-46d9-b307-58c1680ec4c5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:09.803638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:09.832977 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 396/801] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:09:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:09:09.833832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91382f-976a-453f-bbf0-5cd63ab3602d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:09.863880 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb596ff8-a942-46d9-b307-58c1680ec4c5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:09.883917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:09.899384 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc83dc-391d-4a87-8206-1b990a261c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:09.899970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc83dc-391d-4a87-8206-1b990a261c26 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:09.900252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc83dc-391d-4a87-8206-1b990a261c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:09.900521 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc83dc-391d-4a87-8206-1b990a261c26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:09.911114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc83dc-391d-4a87-8206-1b990a261c26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:09.913115 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc83dc-391d-4a87-8206-1b990a261c26 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:09.926963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-cfbc83dc-391d-4a87-8206-1b990a261c26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:09.926963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:09.926963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:09.926963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:09.926963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:09.926963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:09.926963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:09.926963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:09.926963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:09.926963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:09.926963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:09.926963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:09.926963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:09.926963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:09.926963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:09.926963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:09.928057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:09.929532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:09.929532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:09.929532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:09.929532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 406/802] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:09.930034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7926d-6b90-44c4-8b77-1487376a1431 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:09.930305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7926d-6b90-44c4-8b77-1487376a1431 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:09.930562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7926d-6b90-44c4-8b77-1487376a1431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:09.944190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91382f-976a-453f-bbf0-5cd63ab3602d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:09.948250 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-54b7926d-6b90-44c4-8b77-1487376a1431 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:09.949475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-54b7926d-6b90-44c4-8b77-1487376a1431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:09.977437 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb749c31-8de2-4194-91f5-ba047408f9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:09.978043 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb749c31-8de2-4194-91f5-ba047408f9c6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:09.978321 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb749c31-8de2-4194-91f5-ba047408f9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:09.978595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb749c31-8de2-4194-91f5-ba047408f9c6 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:09.979211 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb749c31-8de2-4194-91f5-ba047408f9c6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:09.979524 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb749c31-8de2-4194-91f5-ba047408f9c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:09.980888 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb749c31-8de2-4194-91f5-ba047408f9c6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:09.985164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 397/803] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Oct 6 12:09:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:09.985927 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb749c31-8de2-4194-91f5-ba047408f9c6 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:10.031533 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 407/804] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:10.031533 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122e2bd-c026-4e11-944b-d82b716472b7 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:10.031533 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122e2bd-c026-4e11-944b-d82b716472b7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:10.031533 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122e2bd-c026-4e11-944b-d82b716472b7 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:10.031533 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 408/805] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:10.031533 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122e2bd-c026-4e11-944b-d82b716472b7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:10.060280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:10.100227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb749c31-8de2-4194-91f5-ba047408f9c6 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:10.110632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:10.138104 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac54aa-0bdf-4a9e-87e3-f7947c3c1d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:10.138753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac54aa-0bdf-4a9e-87e3-f7947c3c1d29 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:10.139033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac54aa-0bdf-4a9e-87e3-f7947c3c1d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:10.139331 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac54aa-0bdf-4a9e-87e3-f7947c3c1d29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:10.152240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac54aa-0bdf-4a9e-87e3-f7947c3c1d29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:10.153631 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac54aa-0bdf-4a9e-87e3-f7947c3c1d29 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:10.169686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-8bac54aa-0bdf-4a9e-87e3-f7947c3c1d29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:10.169686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:10.169686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:10.169686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:10.169686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:10.169686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:10.169686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:10.169686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:10.169686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:10.169686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:10.169686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:10.169686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:10.169686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:10.169686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:10.169686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:10.169686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:10.170793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:10.172189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:10.172189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:10.172189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:10.172189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 398/806] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:10.172189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac54aa-0bdf-4a9e-87e3-f7947c3c1d29 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:10.258608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122e2bd-c026-4e11-944b-d82b716472b7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:10.277919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac54aa-0bdf-4a9e-87e3-f7947c3c1d29 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:10.295708 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe47a20-33f5-490f-b069-c419828f6cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:10.296311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe47a20-33f5-490f-b069-c419828f6cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:10.296589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe47a20-33f5-490f-b069-c419828f6cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:10.296862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe47a20-33f5-490f-b069-c419828f6cd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:10.297497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe47a20-33f5-490f-b069-c419828f6cd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:10.297800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe47a20-33f5-490f-b069-c419828f6cd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:10.299158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe47a20-33f5-490f-b069-c419828f6cd1 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:10.310875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 409/807] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:10.311977 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dff4a0-527a-49dd-bea5-ec80b5598955 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:10.312263 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dff4a0-527a-49dd-bea5-ec80b5598955 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:10.312364 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6eb3b-9354-4e80-8f7f-8b7f2d75fc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:10.312764 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dff4a0-527a-49dd-bea5-ec80b5598955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:10.312967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6eb3b-9354-4e80-8f7f-8b7f2d75fc69 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:10.313244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6eb3b-9354-4e80-8f7f-8b7f2d75fc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:10.313513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6eb3b-9354-4e80-8f7f-8b7f2d75fc69 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:10.314128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6eb3b-9354-4e80-8f7f-8b7f2d75fc69 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:10.314428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6eb3b-9354-4e80-8f7f-8b7f2d75fc69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:10.315813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6eb3b-9354-4e80-8f7f-8b7f2d75fc69 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:10.321229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 399/808] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Oct 6 12:09:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:10.321878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6eb3b-9354-4e80-8f7f-8b7f2d75fc69 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:10.330811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d5dff4a0-527a-49dd-bea5-ec80b5598955 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:10.332034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d5dff4a0-527a-49dd-bea5-ec80b5598955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:10.346960 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:10.397930 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 410/809] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:10.398756 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dff4a0-527a-49dd-bea5-ec80b5598955 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:10.422711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6eb3b-9354-4e80-8f7f-8b7f2d75fc69 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:10.432138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:10.456510 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91718a5c-804e-4649-b647-35e971700957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:10.457248 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91718a5c-804e-4649-b647-35e971700957 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:10.457630 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91718a5c-804e-4649-b647-35e971700957 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:10.457997 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91718a5c-804e-4649-b647-35e971700957 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:10.458719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91718a5c-804e-4649-b647-35e971700957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:10.459113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91718a5c-804e-4649-b647-35e971700957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:10.461036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91718a5c-804e-4649-b647-35e971700957 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:10.477052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 400/810] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:10.477052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91718a5c-804e-4649-b647-35e971700957 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:10.517483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dff4a0-527a-49dd-bea5-ec80b5598955 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:10.554160 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6ded4-a554-485a-8a44-316c347eb5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:10.554160 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6ded4-a554-485a-8a44-316c347eb5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:10.554160 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6ded4-a554-485a-8a44-316c347eb5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:10.554160 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6ded4-a554-485a-8a44-316c347eb5a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:10.560819 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:10.567700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6ded4-a554-485a-8a44-316c347eb5a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:10.573031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-05c6ded4-a554-485a-8a44-316c347eb5a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:09:10.573031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:10.573031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:10.573031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:10.573031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:10.573031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:10.573031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:10.573031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:10.573031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:10.573031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:10.573031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:10.573031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:10.573031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:10.573031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:10.573031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:10.573031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:10.573996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:10.573996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:10.573996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:10.573996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:10.573996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:10.573996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:10.573996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:10.573996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:10.573996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:10.573996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:10.573996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:10.573996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:10.573996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:10.573996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:10.573996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:10.573996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:10.574864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:10.574864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:10.574864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:10.574864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:10.574864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:10.574864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:10.574864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:10.574864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:10.574864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:10.574864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:10.574864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:09:10.574864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:10.574864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 411/811] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:09:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:10.576311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6ded4-a554-485a-8a44-316c347eb5a5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:10.584219 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91718a5c-804e-4649-b647-35e971700957 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:10.620608 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd2ce1-6478-4e11-950d-bff1da6aa692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:10.621489 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd2ce1-6478-4e11-950d-bff1da6aa692 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:10.621771 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd2ce1-6478-4e11-950d-bff1da6aa692 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:10.622044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd2ce1-6478-4e11-950d-bff1da6aa692 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:10.628541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fd2ce1-6478-4e11-950d-bff1da6aa692 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:10.629979 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fd2ce1-6478-4e11-950d-bff1da6aa692 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:10.641491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-32fd2ce1-6478-4e11-950d-bff1da6aa692 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:10.641491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:10.641491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:10.641491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:10.641491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:10.641491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:10.641491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:10.641491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:10.641491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:10.641491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:10.641491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:10.641491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:10.641491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:09:10.641491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:09:10.641491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:10.642415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:10.643329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:09:10.643329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:09:10.643329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:09:10.643329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:09:10.643329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:09:10.643329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:09:10.643329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:10.643329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:10.647488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 401/812] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:10.648333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd2ce1-6478-4e11-950d-bff1da6aa692 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:10.663930 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:10.681807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6ded4-a554-485a-8a44-316c347eb5a5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:10.715495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87ce38-cfea-4a37-965f-31815d32c519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:10.716111 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87ce38-cfea-4a37-965f-31815d32c519 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:10.716390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87ce38-cfea-4a37-965f-31815d32c519 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:10.716663 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87ce38-cfea-4a37-965f-31815d32c519 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:10.717316 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87ce38-cfea-4a37-965f-31815d32c519 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:10.718664 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87ce38-cfea-4a37-965f-31815d32c519 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:10.751025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 412/813] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:09:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:09:10.751025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87ce38-cfea-4a37-965f-31815d32c519 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:10.751368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd2ce1-6478-4e11-950d-bff1da6aa692 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:10.794655 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc1f1e-10c2-412b-b880-b7cbc56cafb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:10.795315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc1f1e-10c2-412b-b880-b7cbc56cafb5 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:10.795660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc1f1e-10c2-412b-b880-b7cbc56cafb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:10.795936 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc1f1e-10c2-412b-b880-b7cbc56cafb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:10.803665 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc1f1e-10c2-412b-b880-b7cbc56cafb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:10.805175 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc1f1e-10c2-412b-b880-b7cbc56cafb5 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:10.808193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:10.820586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-fffc1f1e-10c2-412b-b880-b7cbc56cafb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:10.820586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:10.820586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:10.820586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:10.820586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:10.820586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:10.820586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:10.820586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:10.820586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:10.820586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:10.820586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:10.820586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:10.820586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:09:10.820586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:09:10.820586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:10.821646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:10.822541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:09:10.822541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:09:10.822541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:09:10.822541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:09:10.822541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:09:10.822541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:09:10.822541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:10.822541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:10.822541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 402/814] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:10.823042 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc1f1e-10c2-412b-b880-b7cbc56cafb5 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:10.873862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87ce38-cfea-4a37-965f-31815d32c519 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:10.910651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5b029-270b-44ef-9a02-fabf0c238571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:10.911268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5b029-270b-44ef-9a02-fabf0c238571 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:10.911574 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5b029-270b-44ef-9a02-fabf0c238571 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:10.911854 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5b029-270b-44ef-9a02-fabf0c238571 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:10.912552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5b029-270b-44ef-9a02-fabf0c238571 None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:10.912859 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5b029-270b-44ef-9a02-fabf0c238571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:10.914220 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5b029-270b-44ef-9a02-fabf0c238571 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:10.926395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 413/815] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:09:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:10.927727 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5b029-270b-44ef-9a02-fabf0c238571 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:10.931913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:10.939931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc1f1e-10c2-412b-b880-b7cbc56cafb5 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:10.980126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda98e1-c865-429d-b267-c409e5f892a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:10.980908 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda98e1-c865-429d-b267-c409e5f892a1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:10.981192 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda98e1-c865-429d-b267-c409e5f892a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:10.981466 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda98e1-c865-429d-b267-c409e5f892a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:10.982153 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda98e1-c865-429d-b267-c409e5f892a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:10.982459 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda98e1-c865-429d-b267-c409e5f892a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:10.983913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda98e1-c865-429d-b267-c409e5f892a1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:10.996299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 403/816] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:09:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:10.999819 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda98e1-c865-429d-b267-c409e5f892a1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:11.045878 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5b029-270b-44ef-9a02-fabf0c238571 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:11.063150 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:11.085909 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca3e10-1d5a-4c0a-a4b0-8ca7e77db1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:11.086820 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca3e10-1d5a-4c0a-a4b0-8ca7e77db1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:11.087272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca3e10-1d5a-4c0a-a4b0-8ca7e77db1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:11.087782 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca3e10-1d5a-4c0a-a4b0-8ca7e77db1d3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:11.088534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca3e10-1d5a-4c0a-a4b0-8ca7e77db1d3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:11.090068 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca3e10-1d5a-4c0a-a4b0-8ca7e77db1d3 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-c5ca3e10-1d5a-4c0a-a4b0-8ca7e77db1d3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:11.099573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:11.101123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:11.101123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:11.101123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:11.101123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:11.101123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:11.101123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:11.101123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:11.101123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:11.101123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:11.101123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:11.101123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:11.101123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:11.101123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:11.101123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:11.101123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:11.101123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:11.101995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:11.101995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:11.101995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:11.101995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:11.101995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:11.101995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:11.101995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:11.101995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:11.101995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:11.101995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:11.101995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:11.101995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:11.101995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 12:09:11.101995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:11.103997 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 414/817] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:10 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:11.104906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca3e10-1d5a-4c0a-a4b0-8ca7e77db1d3 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:11.111783 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda98e1-c865-429d-b267-c409e5f892a1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:11.150030 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88325995-137d-4e82-b949-ee8de813f845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:11.151730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88325995-137d-4e82-b949-ee8de813f845 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:11.151730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88325995-137d-4e82-b949-ee8de813f845 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:11.151730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88325995-137d-4e82-b949-ee8de813f845 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:11.152649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88325995-137d-4e82-b949-ee8de813f845 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:11.153206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88325995-137d-4e82-b949-ee8de813f845 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:11.155032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88325995-137d-4e82-b949-ee8de813f845 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:11.167137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 404/818] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:09:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:11.167932 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88325995-137d-4e82-b949-ee8de813f845 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:11.183311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:11.232359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca3e10-1d5a-4c0a-a4b0-8ca7e77db1d3 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:11.274437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7ffaf-e572-490f-9119-5069a31d6ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:11.275376 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7ffaf-e572-490f-9119-5069a31d6ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:11.275858 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7ffaf-e572-490f-9119-5069a31d6ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:11.276134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7ffaf-e572-490f-9119-5069a31d6ae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:11.277986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88325995-137d-4e82-b949-ee8de813f845 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:11.288006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7ffaf-e572-490f-9119-5069a31d6ae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:11.289454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7ffaf-e572-490f-9119-5069a31d6ae9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:11.299793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:11.303449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-efa7ffaf-e572-490f-9119-5069a31d6ae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:11.303449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:11.303449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:11.303449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:11.303449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:11.303449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:11.303449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:11.303449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:11.303449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:11.303449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:11.303449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:11.303449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:11.303449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:11.303449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:11.303449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:11.303449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:11.304937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:11.306110 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:11.306110 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:11.306110 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:11.309251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 415/819] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:11.310098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7ffaf-e572-490f-9119-5069a31d6ae9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:11.317531 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c6be8-4af5-4c76-87bc-ec24eac8c12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:11.319359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c6be8-4af5-4c76-87bc-ec24eac8c12c None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:11.319696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c6be8-4af5-4c76-87bc-ec24eac8c12c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:11.319970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c6be8-4af5-4c76-87bc-ec24eac8c12c None admin] PATH_INFO: `/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/3deb4b248f884c19bf1e444f2fe0923a/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:11.322032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c6be8-4af5-4c76-87bc-ec24eac8c12c None admin] RBAC: Authorizing `identity:create_grant(project_id=314d3d13ba2e4a0f93f5d3f34c50a96f, user_id=3deb4b248f884c19bf1e444f2fe0923a, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:11.323645 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c6be8-4af5-4c76-87bc-ec24eac8c12c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:11.347549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 405/820] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:09:11 2022] PUT /identity/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/3deb4b248f884c19bf1e444f2fe0923a/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:09:11.347549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c6be8-4af5-4c76-87bc-ec24eac8c12c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:11.432934 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7ffaf-e572-490f-9119-5069a31d6ae9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:11.454495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:11.475213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c6be8-4af5-4c76-87bc-ec24eac8c12c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:11.475459 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a9c60-5dca-48b3-854f-ca4dab18b2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:11.476235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a9c60-5dca-48b3-854f-ca4dab18b2cf None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:11.476613 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a9c60-5dca-48b3-854f-ca4dab18b2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:11.477083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a9c60-5dca-48b3-854f-ca4dab18b2cf None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:11.477982 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a9c60-5dca-48b3-854f-ca4dab18b2cf None admin] RBAC: Adding query filter params (name=network) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:11.478383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a9c60-5dca-48b3-854f-ca4dab18b2cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:11.479900 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a9c60-5dca-48b3-854f-ca4dab18b2cf None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:11.490325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 416/821] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Oct 6 12:09:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:11.492455 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a9c60-5dca-48b3-854f-ca4dab18b2cf None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:11.521736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947ad4f-b087-4e03-918d-9714f1bc69dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:11.523278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947ad4f-b087-4e03-918d-9714f1bc69dd None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:11.523729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947ad4f-b087-4e03-918d-9714f1bc69dd None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:11.524223 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947ad4f-b087-4e03-918d-9714f1bc69dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:11.525094 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947ad4f-b087-4e03-918d-9714f1bc69dd None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=5a46e63d34564d34b6b5752706375798) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:11.525568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947ad4f-b087-4e03-918d-9714f1bc69dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:11.527342 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947ad4f-b087-4e03-918d-9714f1bc69dd None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:11.535067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 406/822] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:09:11 2022] GET /identity/v3/role_assignments?user.id=5a46e63d34564d34b6b5752706375798&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:11.536184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947ad4f-b087-4e03-918d-9714f1bc69dd None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:11.609172 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:11.638695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a9c60-5dca-48b3-854f-ca4dab18b2cf None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:11.679746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947ad4f-b087-4e03-918d-9714f1bc69dd None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:11.686090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746924b-525d-4ac6-8028-5055689b21dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:11.686970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746924b-525d-4ac6-8028-5055689b21dc None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:11.687395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746924b-525d-4ac6-8028-5055689b21dc None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:11.687837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746924b-525d-4ac6-8028-5055689b21dc None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:11.689393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746924b-525d-4ac6-8028-5055689b21dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:11.689848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746924b-525d-4ac6-8028-5055689b21dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:11.691825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746924b-525d-4ac6-8028-5055689b21dc None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:11.707498 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 417/823] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:11.718497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746924b-525d-4ac6-8028-5055689b21dc None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:11.726487 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592acd32-d7e3-43d6-9e33-b9352c073afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:11.727965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592acd32-d7e3-43d6-9e33-b9352c073afa None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:11.727965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592acd32-d7e3-43d6-9e33-b9352c073afa None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:11.727965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592acd32-d7e3-43d6-9e33-b9352c073afa None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:11.728530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592acd32-d7e3-43d6-9e33-b9352c073afa None admin] RBAC: Adding query filter params (type=network) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:11.728931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592acd32-d7e3-43d6-9e33-b9352c073afa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:11.730478 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592acd32-d7e3-43d6-9e33-b9352c073afa None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:11.739480 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 407/824] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Oct 6 12:09:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:11.745619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592acd32-d7e3-43d6-9e33-b9352c073afa None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:11.808089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:11.878249 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592acd32-d7e3-43d6-9e33-b9352c073afa None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:11.888148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:11.890876 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746924b-525d-4ac6-8028-5055689b21dc None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:11.917178 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47567bdc-8901-432a-923a-53d99b26ea4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:11.917797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47567bdc-8901-432a-923a-53d99b26ea4f None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:11.918077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47567bdc-8901-432a-923a-53d99b26ea4f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:11.918351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47567bdc-8901-432a-923a-53d99b26ea4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:11.919014 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47567bdc-8901-432a-923a-53d99b26ea4f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:11.920671 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47567bdc-8901-432a-923a-53d99b26ea4f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:11.936812 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b066e-7468-4e73-919e-9902fa4a0a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:11.937534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b066e-7468-4e73-919e-9902fa4a0a0a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:11.937769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b066e-7468-4e73-919e-9902fa4a0a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:11.938164 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b066e-7468-4e73-919e-9902fa4a0a0a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:11.946909 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 408/825] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:09:11 2022] POST /identity/v3/endpoints => generated 337 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:09:11.957593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b066e-7468-4e73-919e-9902fa4a0a0a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:11.960087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b066e-7468-4e73-919e-9902fa4a0a0a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:11.980311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-065b066e-7468-4e73-919e-9902fa4a0a0a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 12:09:11.980311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:11.980311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:11.980311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:11.980311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:11.980311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:11.980311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:11.980311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:11.980311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:11.980311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:11.980311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:11.980311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:11.980311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:09:11.980311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:09:11.980311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:09:11.980311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:11.981549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:11.982643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:11.982643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:11.982643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:09:11.982643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:09:11.982643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:09:11.982643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:09:11.982643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:09:11.982643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:09:11.982643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 12:09:11.982643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:11.984751 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 418/826] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Oct 6 12:09:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:11.991493 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47567bdc-8901-432a-923a-53d99b26ea4f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:12.118593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47567bdc-8901-432a-923a-53d99b26ea4f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:12.166984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57218790-6d6d-4a41-8a35-3293948a5499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:12.167873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57218790-6d6d-4a41-8a35-3293948a5499 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:12.168301 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57218790-6d6d-4a41-8a35-3293948a5499 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:12.168723 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57218790-6d6d-4a41-8a35-3293948a5499 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:12.169634 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57218790-6d6d-4a41-8a35-3293948a5499 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:12.170084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57218790-6d6d-4a41-8a35-3293948a5499 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:12.171680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57218790-6d6d-4a41-8a35-3293948a5499 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:12.194728 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 409/827] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Oct 6 12:09:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:12.200571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b066e-7468-4e73-919e-9902fa4a0a0a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:12.313117 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b066e-7468-4e73-919e-9902fa4a0a0a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:12.315681 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:12.354119 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c35abf-d4b3-466d-9cf3-ca2261e5fc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:12.354856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c35abf-d4b3-466d-9cf3-ca2261e5fc9c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:12.355135 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c35abf-d4b3-466d-9cf3-ca2261e5fc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:12.355406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c35abf-d4b3-466d-9cf3-ca2261e5fc9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:12.368330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c35abf-d4b3-466d-9cf3-ca2261e5fc9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:12.369768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c35abf-d4b3-466d-9cf3-ca2261e5fc9c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:12.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-39c35abf-d4b3-466d-9cf3-ca2261e5fc9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:12.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:12.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:12.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:12.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:12.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:12.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:12.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:12.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:12.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:12.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:12.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:12.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:12.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:12.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:12.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:12.386428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:12.387355 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:12.387355 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:12.387355 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:12.391248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 419/828] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:12.392311 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57218790-6d6d-4a41-8a35-3293948a5499 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:12.419975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:12.525353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57218790-6d6d-4a41-8a35-3293948a5499 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:12.560913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b507e6-0ef1-497e-ab23-9444fe0e2038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:12.561525 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b507e6-0ef1-497e-ab23-9444fe0e2038 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:12.561806 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b507e6-0ef1-497e-ab23-9444fe0e2038 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:12.562078 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b507e6-0ef1-497e-ab23-9444fe0e2038 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:12.562714 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b507e6-0ef1-497e-ab23-9444fe0e2038 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:12.563014 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b507e6-0ef1-497e-ab23-9444fe0e2038 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:12.564397 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b507e6-0ef1-497e-ab23-9444fe0e2038 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:12.576056 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 410/829] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:12.581129 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c35abf-d4b3-466d-9cf3-ca2261e5fc9c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:12.683408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c35abf-d4b3-466d-9cf3-ca2261e5fc9c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:12.718173 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f69e88-f058-4caa-bdb5-f446bcdbe22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:12.718803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f69e88-f058-4caa-bdb5-f446bcdbe22f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:12.719080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f69e88-f058-4caa-bdb5-f446bcdbe22f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:12.719351 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f69e88-f058-4caa-bdb5-f446bcdbe22f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:12.727027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f69e88-f058-4caa-bdb5-f446bcdbe22f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:12.728527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f69e88-f058-4caa-bdb5-f446bcdbe22f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:12.739790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-18f69e88-f058-4caa-bdb5-f446bcdbe22f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:12.739790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:12.739790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:12.739790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:12.739790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:12.739790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:12.739790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:12.739790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:12.739790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:12.739790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:12.739790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:12.739790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:12.739790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:09:12.739790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:09:12.739790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:12.740877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:12.741821 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:09:12.741821 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:09:12.741821 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:09:12.741821 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:09:12.741821 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:09:12.741821 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:09:12.741821 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:12.741821 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:12.741821 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 420/830] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:12.747068 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b507e6-0ef1-497e-ab23-9444fe0e2038 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:12.832518 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:12.856296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b507e6-0ef1-497e-ab23-9444fe0e2038 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:12.892677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928a4dc-7c0c-406e-abca-a3194f7edc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:12.893293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928a4dc-7c0c-406e-abca-a3194f7edc67 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:12.893573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928a4dc-7c0c-406e-abca-a3194f7edc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:12.893846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928a4dc-7c0c-406e-abca-a3194f7edc67 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:12.894542 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928a4dc-7c0c-406e-abca-a3194f7edc67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:12.894844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928a4dc-7c0c-406e-abca-a3194f7edc67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:12.896321 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928a4dc-7c0c-406e-abca-a3194f7edc67 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:12.911668 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 411/831] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:09:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:12.917581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f69e88-f058-4caa-bdb5-f446bcdbe22f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:12.937420 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:13.023598 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f69e88-f058-4caa-bdb5-f446bcdbe22f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:13.031634 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ad576-b2c9-486f-ab76-827b9ce7bb17 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:13.031930 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ad576-b2c9-486f-ab76-827b9ce7bb17 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:13.032210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ad576-b2c9-486f-ab76-827b9ce7bb17 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:13.034840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 412/832] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:13.038764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebabbe8-2a88-4729-b6bf-9a194b4b514e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:13.039050 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebabbe8-2a88-4729-b6bf-9a194b4b514e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:13.039325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebabbe8-2a88-4729-b6bf-9a194b4b514e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:13.060299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-5ebabbe8-2a88-4729-b6bf-9a194b4b514e None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:13.061502 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5ebabbe8-2a88-4729-b6bf-9a194b4b514e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:13.063067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480eb646-4ea3-4878-badf-95be499b68e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:13.063751 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480eb646-4ea3-4878-badf-95be499b68e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:13.064027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480eb646-4ea3-4878-badf-95be499b68e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:13.064307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480eb646-4ea3-4878-badf-95be499b68e8 None admin] PATH_INFO: `/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/96a302378dc0416d87c1cf11a3ba0a9c/roles/910821ade2904d23b75fa69e040fb82d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:13.078414 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480eb646-4ea3-4878-badf-95be499b68e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=314d3d13ba2e4a0f93f5d3f34c50a96f, user_id=96a302378dc0416d87c1cf11a3ba0a9c, role_id=910821ade2904d23b75fa69e040fb82d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:13.079876 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480eb646-4ea3-4878-badf-95be499b68e8 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:13.094077 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 421/833] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:09:12 2022] PUT /identity/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/96a302378dc0416d87c1cf11a3ba0a9c/roles/910821ade2904d23b75fa69e040fb82d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:09:13.151720 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 413/834] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:13.430985 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480eb646-4ea3-4878-badf-95be499b68e8 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:13.496658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:13.535144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480eb646-4ea3-4878-badf-95be499b68e8 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:13.572152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784687d2-a4f6-4580-b5bb-4abe0b317d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:13.572782 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784687d2-a4f6-4580-b5bb-4abe0b317d15 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:13.573062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784687d2-a4f6-4580-b5bb-4abe0b317d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:13.573335 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784687d2-a4f6-4580-b5bb-4abe0b317d15 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:13.573919 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784687d2-a4f6-4580-b5bb-4abe0b317d15 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:13.575278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784687d2-a4f6-4580-b5bb-4abe0b317d15 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-784687d2-a4f6-4580-b5bb-4abe0b317d15 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:13.584383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:13.585388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:13.585388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:13.585388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:13.585388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:13.585388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:13.585388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:13.585388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:13.585388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:13.585388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:13.585388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:13.585388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:13.585388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:13.585388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:13.585388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:13.585388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:13.585388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:13.586249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:13.586249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:13.586249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:13.586249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:13.586249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:13.586249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:13.586249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:13.586249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:13.586249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:13.586249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:13.586249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:13.586249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:13.586249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 12:09:13.586249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:13.589501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 422/835] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:09:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:13.592027 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebabbe8-2a88-4729-b6bf-9a194b4b514e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:13.661780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:13.698863 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebabbe8-2a88-4729-b6bf-9a194b4b514e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:13.736711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2695885-9707-4630-b3a9-b57e604253ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:13.737321 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2695885-9707-4630-b3a9-b57e604253ba None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:13.737605 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2695885-9707-4630-b3a9-b57e604253ba None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:13.737876 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2695885-9707-4630-b3a9-b57e604253ba None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:13.738503 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2695885-9707-4630-b3a9-b57e604253ba None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:13.738807 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2695885-9707-4630-b3a9-b57e604253ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:13.740264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2695885-9707-4630-b3a9-b57e604253ba None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:13.750009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 414/836] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:09:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:13.751406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784687d2-a4f6-4580-b5bb-4abe0b317d15 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:13.811705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88a8ed-dab5-464a-bd0d-cd98e28ecd4b None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:13.812019 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88a8ed-dab5-464a-bd0d-cd98e28ecd4b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:13.812260 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88a8ed-dab5-464a-bd0d-cd98e28ecd4b None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:13.813862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 415/837] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:13.819107 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51355a5-094d-4a87-b906-99607d34d568 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:13.819393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51355a5-094d-4a87-b906-99607d34d568 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:13.819712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51355a5-094d-4a87-b906-99607d34d568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:13.838520 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a51355a5-094d-4a87-b906-99607d34d568 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:13.839740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a51355a5-094d-4a87-b906-99607d34d568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:13.864085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784687d2-a4f6-4580-b5bb-4abe0b317d15 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:13.901800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:13.917166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867909e6-370f-4d6a-8625-c67e2a284530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:13.918270 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867909e6-370f-4d6a-8625-c67e2a284530 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:13.918561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867909e6-370f-4d6a-8625-c67e2a284530 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:13.918838 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867909e6-370f-4d6a-8625-c67e2a284530 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:13.919770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867909e6-370f-4d6a-8625-c67e2a284530 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:13.920085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867909e6-370f-4d6a-8625-c67e2a284530 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:13.921680 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867909e6-370f-4d6a-8625-c67e2a284530 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:13.931578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 416/838] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:13.939632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 423/839] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:09:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:14.194830 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51355a5-094d-4a87-b906-99607d34d568 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:14.248836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24944122-b1a0-4b57-84af-684594695dc4 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:14.249121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24944122-b1a0-4b57-84af-684594695dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:14.249399 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24944122-b1a0-4b57-84af-684594695dc4 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:14.251784 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 424/840] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:14.257578 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fad4ea-7ddc-43e3-ab39-f54bad81bd1b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:14.257862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fad4ea-7ddc-43e3-ab39-f54bad81bd1b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:14.258235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fad4ea-7ddc-43e3-ab39-f54bad81bd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:14.261334 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:14.279776 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f4fad4ea-7ddc-43e3-ab39-f54bad81bd1b None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:14.280962 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f4fad4ea-7ddc-43e3-ab39-f54bad81bd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:14.301872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51355a5-094d-4a87-b906-99607d34d568 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:14.337660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588c38c-c076-4544-94af-87995ec8f790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:14.338293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588c38c-c076-4544-94af-87995ec8f790 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:14.338574 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588c38c-c076-4544-94af-87995ec8f790 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:14.338845 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588c38c-c076-4544-94af-87995ec8f790 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:14.350633 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588c38c-c076-4544-94af-87995ec8f790 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:14.355916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-5588c38c-c076-4544-94af-87995ec8f790 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:09:14.355916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:14.355916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:14.355916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:14.355916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:14.355916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:14.355916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:14.355916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:14.355916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:14.355916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:14.355916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:14.355916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:14.355916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:14.355916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:14.355916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:14.355916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:14.356879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:14.356879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:14.356879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:14.356879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:14.356879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:14.356879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:14.356879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:14.356879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:14.356879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:14.356879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:14.356879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:14.356879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:14.356879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:14.356879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:14.356879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:14.356879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:14.357743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:14.357743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:14.357743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:14.357743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:14.357743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:14.357743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:14.357743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:14.357743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:14.357743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:14.357743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:14.357743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:09:14.357743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:14.357743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 417/841] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:09:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:14.364765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 425/842] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:14.364906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588c38c-c076-4544-94af-87995ec8f790 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:14.473666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588c38c-c076-4544-94af-87995ec8f790 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:14.497147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:14.509355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de507f28-81c1-4053-b741-de23549bc1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:14.509966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de507f28-81c1-4053-b741-de23549bc1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:14.510656 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de507f28-81c1-4053-b741-de23549bc1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:14.510933 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de507f28-81c1-4053-b741-de23549bc1d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:14.511631 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de507f28-81c1-4053-b741-de23549bc1d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:14.511941 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de507f28-81c1-4053-b741-de23549bc1d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:14.513377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de507f28-81c1-4053-b741-de23549bc1d9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:14.522487 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 418/843] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:09:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:14.528238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fad4ea-7ddc-43e3-ab39-f54bad81bd1b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:14.629305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fad4ea-7ddc-43e3-ab39-f54bad81bd1b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:14.644220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de507f28-81c1-4053-b741-de23549bc1d9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:14.654910 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:14.672282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6a0b9-a87d-4989-83ad-61ec68746bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:14.673327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6a0b9-a87d-4989-83ad-61ec68746bdb None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:14.673811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6a0b9-a87d-4989-83ad-61ec68746bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:14.674423 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6a0b9-a87d-4989-83ad-61ec68746bdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:14.686726 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6a0b9-a87d-4989-83ad-61ec68746bdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:14.688314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6a0b9-a87d-4989-83ad-61ec68746bdb None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:14.706569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-a5a6a0b9-a87d-4989-83ad-61ec68746bdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:14.706569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:14.706569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:14.706569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:14.706569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:14.706569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:14.706569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:14.706569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:14.706569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:14.706569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:14.706569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:14.706569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:14.706569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:14.706569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:14.706569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:14.706569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:14.707638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:14.708568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:14.708568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:14.708568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:14.710278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 426/844] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:14.715466 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6a0b9-a87d-4989-83ad-61ec68746bdb None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:14.778232 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de507f28-81c1-4053-b741-de23549bc1d9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:14.822806 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edd9b5-bd7f-4b3b-a2f9-84c70b20b4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:14.823912 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edd9b5-bd7f-4b3b-a2f9-84c70b20b4de None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:14.824418 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edd9b5-bd7f-4b3b-a2f9-84c70b20b4de None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:14.824896 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edd9b5-bd7f-4b3b-a2f9-84c70b20b4de None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:14.825774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edd9b5-bd7f-4b3b-a2f9-84c70b20b4de None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:14.827492 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edd9b5-bd7f-4b3b-a2f9-84c70b20b4de None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:14.829504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6a0b9-a87d-4989-83ad-61ec68746bdb None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:14.837374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-b7edd9b5-bd7f-4b3b-a2f9-84c70b20b4de None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 12:09:14.837374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:14.837374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:14.837374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:14.837374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:14.837374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:14.837374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:14.837374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:14.837374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 12:09:14.837374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:14.837374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:14.837374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:14.837374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:14.837374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:14.837374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:14.837374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:14.838372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:14.839312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:14.839312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:14.839312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:14.839312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:14.839312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:14.839312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:14.839312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:14.839312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:14.839312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:14.839312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:14.839312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:14.839312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:14.839312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 12:09:14.839312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:14.846901 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 419/845] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Oct 6 12:09:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:14.848381 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edd9b5-bd7f-4b3b-a2f9-84c70b20b4de None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:14.865085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d737ac4-cff5-48a6-aa96-3a1ef930dfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:14.865879 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d737ac4-cff5-48a6-aa96-3a1ef930dfc5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:14.866230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d737ac4-cff5-48a6-aa96-3a1ef930dfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:14.866513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d737ac4-cff5-48a6-aa96-3a1ef930dfc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:14.867186 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d737ac4-cff5-48a6-aa96-3a1ef930dfc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:14.867583 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d737ac4-cff5-48a6-aa96-3a1ef930dfc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:14.869200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d737ac4-cff5-48a6-aa96-3a1ef930dfc5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:14.887817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 427/846] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:14.889738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d737ac4-cff5-48a6-aa96-3a1ef930dfc5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:14.910095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:14.927068 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:14.969218 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edd9b5-bd7f-4b3b-a2f9-84c70b20b4de None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:15.000041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d737ac4-cff5-48a6-aa96-3a1ef930dfc5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:15.007562 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b63c2-e42b-4fff-952d-75588228690a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:15.008460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b63c2-e42b-4fff-952d-75588228690a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:15.008946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b63c2-e42b-4fff-952d-75588228690a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:15.009422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b63c2-e42b-4fff-952d-75588228690a None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:15.010311 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b63c2-e42b-4fff-952d-75588228690a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:15.010819 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b63c2-e42b-4fff-952d-75588228690a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:15.012529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b63c2-e42b-4fff-952d-75588228690a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:15.022420 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 420/847] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Oct 6 12:09:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:15.023909 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b63c2-e42b-4fff-952d-75588228690a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:15.039039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5eed35-d5a3-4e7a-98c0-aec304f706d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:15.039704 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5eed35-d5a3-4e7a-98c0-aec304f706d3 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:15.039984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5eed35-d5a3-4e7a-98c0-aec304f706d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:15.040257 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5eed35-d5a3-4e7a-98c0-aec304f706d3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:15.047993 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5eed35-d5a3-4e7a-98c0-aec304f706d3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:15.049729 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5eed35-d5a3-4e7a-98c0-aec304f706d3 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:15.062581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-4e5eed35-d5a3-4e7a-98c0-aec304f706d3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 12:09:15.062581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:15.062581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:15.062581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:15.062581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:15.062581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:15.062581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:15.062581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:15.062581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:15.062581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:15.062581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:15.062581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:15.062581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:09:15.062581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:09:15.062581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:09:15.062581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:15.063608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:15.064588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:15.064588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:15.064588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:09:15.064588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:09:15.064588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:09:15.064588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:09:15.064588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:09:15.064588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:09:15.064588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 12:09:15.064588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:15.064588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 428/848] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:09:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:15.071654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5eed35-d5a3-4e7a-98c0-aec304f706d3 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:15.138549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b63c2-e42b-4fff-952d-75588228690a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:15.152749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:15.173436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:15.180497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c0e72-b4db-44ec-b8a4-4e4cf3dd99ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:15.180914 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5eed35-d5a3-4e7a-98c0-aec304f706d3 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:15.181625 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c0e72-b4db-44ec-b8a4-4e4cf3dd99ee None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:15.182117 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c0e72-b4db-44ec-b8a4-4e4cf3dd99ee None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:15.182664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c0e72-b4db-44ec-b8a4-4e4cf3dd99ee None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:15.183665 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c0e72-b4db-44ec-b8a4-4e4cf3dd99ee None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:15.184177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c0e72-b4db-44ec-b8a4-4e4cf3dd99ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:15.185822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c0e72-b4db-44ec-b8a4-4e4cf3dd99ee None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:15.200000 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 421/849] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Oct 6 12:09:15 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:15.224437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba33b4-243f-4f68-8d77-f1fd0d364ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:15.225154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba33b4-243f-4f68-8d77-f1fd0d364ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:15.225438 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba33b4-243f-4f68-8d77-f1fd0d364ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:15.225712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba33b4-243f-4f68-8d77-f1fd0d364ff8 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:15.226548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba33b4-243f-4f68-8d77-f1fd0d364ff8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:15.226863 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba33b4-243f-4f68-8d77-f1fd0d364ff8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:15.228553 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba33b4-243f-4f68-8d77-f1fd0d364ff8 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:15.251492 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 429/850] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Oct 6 12:09:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:15.257689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c0e72-b4db-44ec-b8a4-4e4cf3dd99ee None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:15.345344 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f246a8-6a85-4d0b-9973-7535568a4fd1 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:15.345555 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f246a8-6a85-4d0b-9973-7535568a4fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:15.345813 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f246a8-6a85-4d0b-9973-7535568a4fd1 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:15.347400 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 430/851] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:09:15.352937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f08de-376b-479b-9e11-e218a3b4c1a4 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:15.353225 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f08de-376b-479b-9e11-e218a3b4c1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:15.353498 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f08de-376b-479b-9e11-e218a3b4c1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:15.374639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-669f08de-376b-479b-9e11-e218a3b4c1a4 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:15.376010 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-669f08de-376b-479b-9e11-e218a3b4c1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:15.378260 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c0e72-b4db-44ec-b8a4-4e4cf3dd99ee None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:15.428731 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188e940-5208-4c5a-a779-bca9449f5557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:15.429623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188e940-5208-4c5a-a779-bca9449f5557 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:15.430061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188e940-5208-4c5a-a779-bca9449f5557 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:15.430489 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188e940-5208-4c5a-a779-bca9449f5557 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:15.475649 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 431/852] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:15.485386 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:15.510166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188e940-5208-4c5a-a779-bca9449f5557 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:15.511926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188e940-5208-4c5a-a779-bca9449f5557 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:15.617261 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 422/853] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:09:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:09:15.624652 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f08de-376b-479b-9e11-e218a3b4c1a4 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:15.712253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:15.731818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f08de-376b-479b-9e11-e218a3b4c1a4 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:15.766923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349d73a-e5ce-43af-9ac5-7e6075a15cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:15.767578 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349d73a-e5ce-43af-9ac5-7e6075a15cff None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:15.767866 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349d73a-e5ce-43af-9ac5-7e6075a15cff None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:15.768140 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349d73a-e5ce-43af-9ac5-7e6075a15cff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:15.768798 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188e940-5208-4c5a-a779-bca9449f5557 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:15.779665 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349d73a-e5ce-43af-9ac5-7e6075a15cff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:15.781285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349d73a-e5ce-43af-9ac5-7e6075a15cff None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:15.797527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-d349d73a-e5ce-43af-9ac5-7e6075a15cff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:15.797527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:15.797527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:15.797527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:15.797527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:15.797527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:15.797527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:15.797527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:15.797527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:15.797527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:15.797527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:15.797527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:15.797527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:15.797527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:15.797527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:15.797527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:15.798572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:15.800158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:15.800158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:15.800158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:15.800158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 432/854] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:15.804925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349d73a-e5ce-43af-9ac5-7e6075a15cff None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:15.860910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:15.890415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188e940-5208-4c5a-a779-bca9449f5557 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:15.918992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349d73a-e5ce-43af-9ac5-7e6075a15cff None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:15.932098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e93b0f-0e15-40ba-9252-3e93b08d3f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:15.932740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e93b0f-0e15-40ba-9252-3e93b08d3f65 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:15.933020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e93b0f-0e15-40ba-9252-3e93b08d3f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:15.933293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e93b0f-0e15-40ba-9252-3e93b08d3f65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:15.946024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e93b0f-0e15-40ba-9252-3e93b08d3f65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:15.951007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-61e93b0f-0e15-40ba-9252-3e93b08d3f65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:09:15.951007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:15.951007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:15.951007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:15.951007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:15.951007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:15.951007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:15.951007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:15.951007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:15.951007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:15.951007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:15.951007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:15.951007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:15.951007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:15.951007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:15.951007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:15.951986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:15.951986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:15.951986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:15.951986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:15.951986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:15.951986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:15.951986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:15.951986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:15.951986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:15.951986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:15.951986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:15.951986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:15.951986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:15.951986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:15.951986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:15.951986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:15.952872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:15.952872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:15.952872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:15.952872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:15.952872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:15.952872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:15.952872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:15.952872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:15.952872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:15.952872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:15.952872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:09:15.952872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:15.957177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 423/855] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:09:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:15.957991 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e93b0f-0e15-40ba-9252-3e93b08d3f65 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:15.969822 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59d3a3-ea7b-487d-85d6-545abfe59b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:15.971173 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59d3a3-ea7b-487d-85d6-545abfe59b0d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:15.972147 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59d3a3-ea7b-487d-85d6-545abfe59b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:15.972629 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59d3a3-ea7b-487d-85d6-545abfe59b0d None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:15.973761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59d3a3-ea7b-487d-85d6-545abfe59b0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:15.974284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59d3a3-ea7b-487d-85d6-545abfe59b0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:15.976162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59d3a3-ea7b-487d-85d6-545abfe59b0d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:15.978127 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:15.994661 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 433/856] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:16.001356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59d3a3-ea7b-487d-85d6-545abfe59b0d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:16.072754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e93b0f-0e15-40ba-9252-3e93b08d3f65 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:16.112144 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbf512-db67-49f3-89b3-b211a71d73e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:16.112843 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbf512-db67-49f3-89b3-b211a71d73e4 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:16.113127 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbf512-db67-49f3-89b3-b211a71d73e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:16.113402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbf512-db67-49f3-89b3-b211a71d73e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:16.114222 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbf512-db67-49f3-89b3-b211a71d73e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:16.114399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbf512-db67-49f3-89b3-b211a71d73e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:16.116309 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbf512-db67-49f3-89b3-b211a71d73e4 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:16.118850 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:16.129652 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59d3a3-ea7b-487d-85d6-545abfe59b0d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:16.131952 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 424/857] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:09:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:16.133338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbf512-db67-49f3-89b3-b211a71d73e4 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:16.173503 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f5606-c6a8-4fae-8fb2-ce896df739fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:16.174454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f5606-c6a8-4fae-8fb2-ce896df739fc None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:16.174935 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f5606-c6a8-4fae-8fb2-ce896df739fc None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:16.175932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f5606-c6a8-4fae-8fb2-ce896df739fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:16.184712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f5606-c6a8-4fae-8fb2-ce896df739fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:16.186779 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f5606-c6a8-4fae-8fb2-ce896df739fc None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:16.200029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-986f5606-c6a8-4fae-8fb2-ce896df739fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:16.200029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:16.200029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:16.200029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:16.200029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:16.200029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:16.200029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:16.200029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:16.200029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:16.200029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:16.200029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:16.200029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:16.200029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:09:16.200029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:09:16.200029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:16.200924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:16.207121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:09:16.207121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:09:16.207121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:09:16.207121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:09:16.207121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:09:16.207121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:09:16.207121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:16.207121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:16.207121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 434/858] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:16.207121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75ae00-2761-44f6-9172-3a4f670d3c7f None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:16.207121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75ae00-2761-44f6-9172-3a4f670d3c7f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:16.207813 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75ae00-2761-44f6-9172-3a4f670d3c7f None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:16.207813 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 435/859] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:16.207813 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75ae00-2761-44f6-9172-3a4f670d3c7f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:16.245583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbf512-db67-49f3-89b3-b211a71d73e4 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:16.278583 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:16.285772 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5e69a-f3ef-4ef0-ac17-f4646cbf92ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:16.287146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5e69a-f3ef-4ef0-ac17-f4646cbf92ba None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:16.288167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5e69a-f3ef-4ef0-ac17-f4646cbf92ba None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:16.288590 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5e69a-f3ef-4ef0-ac17-f4646cbf92ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:16.299517 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5e69a-f3ef-4ef0-ac17-f4646cbf92ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:16.300970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5e69a-f3ef-4ef0-ac17-f4646cbf92ba None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:16.316292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-22b5e69a-f3ef-4ef0-ac17-f4646cbf92ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:16.316292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:16.316292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:16.316292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:16.316292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:16.316292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:16.316292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:16.316292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:16.316292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:16.316292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:16.316292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:16.316292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:16.316292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:16.316292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:16.316292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:16.316292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:16.317333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:16.318273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:16.318273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:16.318273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:16.318273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 425/860] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:16.319377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21f6c2-049f-4051-85db-b55aed0cbc4e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:16.319377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21f6c2-049f-4051-85db-b55aed0cbc4e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:16.319697 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21f6c2-049f-4051-85db-b55aed0cbc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:16.334829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75ae00-2761-44f6-9172-3a4f670d3c7f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:16.338074 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-5d21f6c2-049f-4051-85db-b55aed0cbc4e None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:16.339258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5d21f6c2-049f-4051-85db-b55aed0cbc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:16.372352 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c0a05-1e2a-4bcb-bbab-f1656d306db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:16.372963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c0a05-1e2a-4bcb-bbab-f1656d306db0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:16.373245 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c0a05-1e2a-4bcb-bbab-f1656d306db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:16.373521 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c0a05-1e2a-4bcb-bbab-f1656d306db0 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:16.374266 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c0a05-1e2a-4bcb-bbab-f1656d306db0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:16.374580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c0a05-1e2a-4bcb-bbab-f1656d306db0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:16.376041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c0a05-1e2a-4bcb-bbab-f1656d306db0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:16.387126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:16.391786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 436/861] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:09:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:16.392673 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c0a05-1e2a-4bcb-bbab-f1656d306db0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:16.415176 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 426/862] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:16.415882 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21f6c2-049f-4051-85db-b55aed0cbc4e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:16.511114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c0a05-1e2a-4bcb-bbab-f1656d306db0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:16.521182 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21f6c2-049f-4051-85db-b55aed0cbc4e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:16.526272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:16.549640 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9ab8b-0165-4a7b-b2c4-772b2b597ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:16.550554 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9ab8b-0165-4a7b-b2c4-772b2b597ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:16.551067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9ab8b-0165-4a7b-b2c4-772b2b597ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:16.551476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9ab8b-0165-4a7b-b2c4-772b2b597ff3 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:16.552330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9ab8b-0165-4a7b-b2c4-772b2b597ff3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:16.552768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9ab8b-0165-4a7b-b2c4-772b2b597ff3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:16.554314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9ab8b-0165-4a7b-b2c4-772b2b597ff3 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:16.556568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c8a19-f615-4ad0-a17e-6f517dde50d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:16.557189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c8a19-f615-4ad0-a17e-6f517dde50d2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:16.557451 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c8a19-f615-4ad0-a17e-6f517dde50d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:16.557721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c8a19-f615-4ad0-a17e-6f517dde50d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:16.558443 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c8a19-f615-4ad0-a17e-6f517dde50d2 None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=3deb4b248f884c19bf1e444f2fe0923a) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:16.558752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c8a19-f615-4ad0-a17e-6f517dde50d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:16.560227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c8a19-f615-4ad0-a17e-6f517dde50d2 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:16.567531 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 437/863] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:16.571697 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 427/864] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:09:16 2022] GET /identity/v3/role_assignments?user.id=3deb4b248f884c19bf1e444f2fe0923a&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:16.579410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9ab8b-0165-4a7b-b2c4-772b2b597ff3 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:16.684491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c8a19-f615-4ad0-a17e-6f517dde50d2 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:16.689501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9ab8b-0165-4a7b-b2c4-772b2b597ff3 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:16.727874 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14569ac9-d564-433c-8b43-5e4976367d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:16.728653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14569ac9-d564-433c-8b43-5e4976367d86 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:16.728998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14569ac9-d564-433c-8b43-5e4976367d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:16.729278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14569ac9-d564-433c-8b43-5e4976367d86 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:16.736151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:16.739410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14569ac9-d564-433c-8b43-5e4976367d86 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:16.742353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14569ac9-d564-433c-8b43-5e4976367d86 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:16.754599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-14569ac9-d564-433c-8b43-5e4976367d86 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 12:09:16.754599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:16.754599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:16.754599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:16.754599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:16.754599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:16.754599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:16.754599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:16.754599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:16.754599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:16.754599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:16.754599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:16.754599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:09:16.754599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:09:16.754599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:09:16.754599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:16.755634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:16.756690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:16.756690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:16.756690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:09:16.756690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:09:16.756690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:09:16.756690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:09:16.756690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:09:16.756690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:09:16.756690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 12:09:16.756690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:16.756690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 438/865] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Oct 6 12:09:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:16.761720 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14569ac9-d564-433c-8b43-5e4976367d86 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:16.808532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:16.813310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c8a19-f615-4ad0-a17e-6f517dde50d2 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:16.853869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238af893-9b00-4fc9-914f-6de36f0c130c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:16.854971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238af893-9b00-4fc9-914f-6de36f0c130c None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:16.855320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238af893-9b00-4fc9-914f-6de36f0c130c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:16.855713 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238af893-9b00-4fc9-914f-6de36f0c130c None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:16.856433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238af893-9b00-4fc9-914f-6de36f0c130c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:16.857872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238af893-9b00-4fc9-914f-6de36f0c130c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:16.883278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 428/866] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Oct 6 12:09:16 2022] POST /identity/v3/services => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:09:16.899768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14569ac9-d564-433c-8b43-5e4976367d86 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:16.956005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dff71c-1911-4c5e-a161-f41dfedf5e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:16.956627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dff71c-1911-4c5e-a161-f41dfedf5e8f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:16.956911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dff71c-1911-4c5e-a161-f41dfedf5e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:16.957186 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dff71c-1911-4c5e-a161-f41dfedf5e8f None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:16.957999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dff71c-1911-4c5e-a161-f41dfedf5e8f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:16.958352 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dff71c-1911-4c5e-a161-f41dfedf5e8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:16.959972 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dff71c-1911-4c5e-a161-f41dfedf5e8f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:16.981514 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 439/867] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Oct 6 12:09:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:16.987460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238af893-9b00-4fc9-914f-6de36f0c130c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:17.103874 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238af893-9b00-4fc9-914f-6de36f0c130c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:17.106317 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:17.140992 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771990af-63bb-4080-abd5-79af552b94fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:17.141612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771990af-63bb-4080-abd5-79af552b94fe None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:17.141894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771990af-63bb-4080-abd5-79af552b94fe None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:17.142170 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771990af-63bb-4080-abd5-79af552b94fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:17.212679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771990af-63bb-4080-abd5-79af552b94fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:17.214272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771990af-63bb-4080-abd5-79af552b94fe None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:17.323233 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 429/868] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:09:16 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:09:17.329577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dff71c-1911-4c5e-a161-f41dfedf5e8f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:17.427217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:17.437271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f8d35-9d54-4e4e-a47d-d8d49cdcf9b5 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:17.437557 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f8d35-9d54-4e4e-a47d-d8d49cdcf9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:17.437832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f8d35-9d54-4e4e-a47d-d8d49cdcf9b5 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:17.439820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 430/869] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:17.443312 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dff71c-1911-4c5e-a161-f41dfedf5e8f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:17.449666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e077857-49e6-4ad5-9448-d1ac37b05c9e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:17.449968 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e077857-49e6-4ad5-9448-d1ac37b05c9e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:17.450241 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e077857-49e6-4ad5-9448-d1ac37b05c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:17.457918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:17.471310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7e077857-49e6-4ad5-9448-d1ac37b05c9e None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:17.472994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7e077857-49e6-4ad5-9448-d1ac37b05c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:17.483318 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90745283-b63b-4328-80f9-f68eb51bf4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:17.483992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90745283-b63b-4328-80f9-f68eb51bf4ea None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:17.484277 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90745283-b63b-4328-80f9-f68eb51bf4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:17.484555 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90745283-b63b-4328-80f9-f68eb51bf4ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:17.496085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90745283-b63b-4328-80f9-f68eb51bf4ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:17.497535 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90745283-b63b-4328-80f9-f68eb51bf4ea None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:17.515885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-90745283-b63b-4328-80f9-f68eb51bf4ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:17.515885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:17.515885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:17.515885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:17.515885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:17.515885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:17.515885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:17.515885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:17.515885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:17.515885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:17.515885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:17.515885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:17.515885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:17.515885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:17.515885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:17.515885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:17.516882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:17.517801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:17.517801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:17.517801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:17.517801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 440/870] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:17.524616 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90745283-b63b-4328-80f9-f68eb51bf4ea None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:17.575530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 431/871] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:17 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:17.646198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90745283-b63b-4328-80f9-f68eb51bf4ea None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:17.682875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530bc96-df7a-458b-8682-b471505a3627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:17.683403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530bc96-df7a-458b-8682-b471505a3627 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:17.683721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530bc96-df7a-458b-8682-b471505a3627 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:17.683995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530bc96-df7a-458b-8682-b471505a3627 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:17.684633 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530bc96-df7a-458b-8682-b471505a3627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:17.684937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530bc96-df7a-458b-8682-b471505a3627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:17.686300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530bc96-df7a-458b-8682-b471505a3627 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:17.695651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:17.699540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 441/872] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:17.705034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e077857-49e6-4ad5-9448-d1ac37b05c9e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:17.806100 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e077857-49e6-4ad5-9448-d1ac37b05c9e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:17.842424 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f33312-0979-4a46-b912-8f69eebe0d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:17.843103 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f33312-0979-4a46-b912-8f69eebe0d88 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:17.843348 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f33312-0979-4a46-b912-8f69eebe0d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:17.843619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f33312-0979-4a46-b912-8f69eebe0d88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:17.845541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:17.851661 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f33312-0979-4a46-b912-8f69eebe0d88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:17.853117 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f33312-0979-4a46-b912-8f69eebe0d88 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:17.856977 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530bc96-df7a-458b-8682-b471505a3627 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:17.865495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-c5f33312-0979-4a46-b912-8f69eebe0d88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:17.865495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:17.865495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:17.865495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:17.865495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:17.865495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:17.865495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:17.865495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:17.865495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:17.865495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:17.865495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:17.865495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:17.865495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:09:17.865495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:09:17.865495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:17.866392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:17.867312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:09:17.867312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:09:17.867312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:09:17.867312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:09:17.867312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:09:17.867312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:09:17.867312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:17.867312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:17.867312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 432/873] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:17.873527 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f33312-0979-4a46-b912-8f69eebe0d88 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:17.993554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f33312-0979-4a46-b912-8f69eebe0d88 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:17.997025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530bc96-df7a-458b-8682-b471505a3627 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:18.037654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efc800-5f6a-41f7-ae6d-12d80d50733c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:18.038273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efc800-5f6a-41f7-ae6d-12d80d50733c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:18.038691 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efc800-5f6a-41f7-ae6d-12d80d50733c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:18.038967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efc800-5f6a-41f7-ae6d-12d80d50733c None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:18.039897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efc800-5f6a-41f7-ae6d-12d80d50733c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:18.040267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efc800-5f6a-41f7-ae6d-12d80d50733c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:18.041766 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efc800-5f6a-41f7-ae6d-12d80d50733c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:18.041939 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97cf2f-6216-41b3-a49e-262a4d16ad19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:18.042666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97cf2f-6216-41b3-a49e-262a4d16ad19 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:18.043051 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97cf2f-6216-41b3-a49e-262a4d16ad19 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:18.043685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97cf2f-6216-41b3-a49e-262a4d16ad19 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:18.044925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97cf2f-6216-41b3-a49e-262a4d16ad19 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:18.046436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97cf2f-6216-41b3-a49e-262a4d16ad19 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:18.057618 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 433/874] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:09:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:18.063596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efc800-5f6a-41f7-ae6d-12d80d50733c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:18.072579 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 442/875] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Oct 6 12:09:17 2022] POST /identity/v3/services => generated 269 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:09:18.135903 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:18.188506 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efc800-5f6a-41f7-ae6d-12d80d50733c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:18.242107 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015dc707-76f1-46b3-9f02-b55998be98a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:18.242733 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015dc707-76f1-46b3-9f02-b55998be98a5 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:18.243549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015dc707-76f1-46b3-9f02-b55998be98a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:18.243549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015dc707-76f1-46b3-9f02-b55998be98a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:18.244137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015dc707-76f1-46b3-9f02-b55998be98a5 None admin] RBAC: Adding query filter params (role.id=910821ade2904d23b75fa69e040fb82d, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=96a302378dc0416d87c1cf11a3ba0a9c) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:18.244454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015dc707-76f1-46b3-9f02-b55998be98a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:18.245852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015dc707-76f1-46b3-9f02-b55998be98a5 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:18.256059 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 434/876] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:09:18 2022] GET /identity/v3/role_assignments?user.id=96a302378dc0416d87c1cf11a3ba0a9c&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=910821ade2904d23b75fa69e040fb82d => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:18.929346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3d16c-3d57-4ff2-ba5a-f43c2ac96ec0 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:18.929868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3d16c-3d57-4ff2-ba5a-f43c2ac96ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:18.929868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3d16c-3d57-4ff2-ba5a-f43c2ac96ec0 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:18.931354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 443/877] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:18.936773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9663eda-ce3b-4897-9494-002e87b996c0 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:18.937187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9663eda-ce3b-4897-9494-002e87b996c0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:18.937354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9663eda-ce3b-4897-9494-002e87b996c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:18.956468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c9663eda-ce3b-4897-9494-002e87b996c0 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:18.957701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c9663eda-ce3b-4897-9494-002e87b996c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:19.032358 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 435/878] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:19.170695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d2098-a42b-4a5c-8087-ca3645fc7320 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:19.171020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d2098-a42b-4a5c-8087-ca3645fc7320 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:19.171243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d2098-a42b-4a5c-8087-ca3645fc7320 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:19.172868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 444/879] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:19.178497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce5f62-3f15-4567-93ce-d343548d15b2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:19.178786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce5f62-3f15-4567-93ce-d343548d15b2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:19.179062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce5f62-3f15-4567-93ce-d343548d15b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:19.198897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-49ce5f62-3f15-4567-93ce-d343548d15b2 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:19.200077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-49ce5f62-3f15-4567-93ce-d343548d15b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:19.244098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:19.278484 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 436/880] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:19.304562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d2098-a42b-4a5c-8087-ca3645fc7320 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:19.366944 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:19.413476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d2098-a42b-4a5c-8087-ca3645fc7320 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:19.449264 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2918874-604b-4c70-906a-402b8f48d630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:19.450227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2918874-604b-4c70-906a-402b8f48d630 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:19.450732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2918874-604b-4c70-906a-402b8f48d630 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:19.451178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2918874-604b-4c70-906a-402b8f48d630 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:19.451959 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2918874-604b-4c70-906a-402b8f48d630 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:19.453543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2918874-604b-4c70-906a-402b8f48d630 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-b2918874-604b-4c70-906a-402b8f48d630 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:19.462984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:19.463998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:19.463998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:19.463998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:19.463998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:19.463998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:19.463998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:19.463998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:19.463998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:19.463998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:19.463998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:19.463998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:19.463998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:19.463998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:19.463998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:19.463998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:19.463998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:19.464870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:19.464870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:19.464870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:19.464870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:19.464870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:19.464870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:19.464870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:19.464870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:19.464870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:19.464870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:19.464870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:19.464870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:19.464870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 12:09:19.464870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:19.467235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 445/881] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Oct 6 12:09:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:19.472908 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce5f62-3f15-4567-93ce-d343548d15b2 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:19.554581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2918874-604b-4c70-906a-402b8f48d630 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:19.573813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce5f62-3f15-4567-93ce-d343548d15b2 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:19.617628 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cc1d0-a411-4353-855e-62d7eedaccfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:19.618244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cc1d0-a411-4353-855e-62d7eedaccfc None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:19.618527 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cc1d0-a411-4353-855e-62d7eedaccfc None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:19.618797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cc1d0-a411-4353-855e-62d7eedaccfc None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:19.619425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cc1d0-a411-4353-855e-62d7eedaccfc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:19.619753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cc1d0-a411-4353-855e-62d7eedaccfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:19.621118 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cc1d0-a411-4353-855e-62d7eedaccfc None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:19.632190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 437/882] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Oct 6 12:09:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:19.633943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cc1d0-a411-4353-855e-62d7eedaccfc None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:19.674785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2918874-604b-4c70-906a-402b8f48d630 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:19.677541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:19.703738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:19.720795 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cef744-f53b-4bca-9df7-aefa31caabf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:19.721434 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cef744-f53b-4bca-9df7-aefa31caabf4 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:19.721712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cef744-f53b-4bca-9df7-aefa31caabf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:19.721980 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cef744-f53b-4bca-9df7-aefa31caabf4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:19.722564 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cef744-f53b-4bca-9df7-aefa31caabf4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:19.724088 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cef744-f53b-4bca-9df7-aefa31caabf4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-67cef744-f53b-4bca-9df7-aefa31caabf4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:19.734255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:19.735268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:19.735268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:19.735268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:19.735268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:19.735268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:19.735268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:19.735268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:19.735268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:19.735268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:19.735268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:19.735268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:19.735268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:19.735268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:19.735268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:19.735268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:19.735268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:19.736513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:19.736513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:19.736513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:19.736513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:19.736513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:19.736513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:19.736513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:19.736513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:19.736513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:19.736513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:19.736513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:19.736513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:19.736513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 12:09:19.736513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:19.740742 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 446/883] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:09:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:19.741575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cef744-f53b-4bca-9df7-aefa31caabf4 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:19.755139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cc1d0-a411-4353-855e-62d7eedaccfc None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:19.793573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9890c3-c603-4a22-9cd6-e35b6da75224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:19.794529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9890c3-c603-4a22-9cd6-e35b6da75224 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:19.795120 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9890c3-c603-4a22-9cd6-e35b6da75224 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:19.795716 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9890c3-c603-4a22-9cd6-e35b6da75224 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:19.796661 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9890c3-c603-4a22-9cd6-e35b6da75224 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:19.797155 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9890c3-c603-4a22-9cd6-e35b6da75224 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:19.798850 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9890c3-c603-4a22-9cd6-e35b6da75224 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:19.810054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 438/884] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Oct 6 12:09:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:19.851892 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cef744-f53b-4bca-9df7-aefa31caabf4 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:19.890617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53498ddf-35e9-404e-b312-857bc4c7627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:19.891244 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53498ddf-35e9-404e-b312-857bc4c7627f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:19.891548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53498ddf-35e9-404e-b312-857bc4c7627f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:19.891830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53498ddf-35e9-404e-b312-857bc4c7627f None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:19.892468 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53498ddf-35e9-404e-b312-857bc4c7627f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:19.892773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53498ddf-35e9-404e-b312-857bc4c7627f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:19.894164 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53498ddf-35e9-404e-b312-857bc4c7627f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:19.900370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 447/885] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:09:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:19.906389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9890c3-c603-4a22-9cd6-e35b6da75224 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:20.018043 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9890c3-c603-4a22-9cd6-e35b6da75224 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:20.019595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:20.056091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529526e-05ca-4b40-9f32-dfd2125b31b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:20.056705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529526e-05ca-4b40-9f32-dfd2125b31b9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:20.056982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529526e-05ca-4b40-9f32-dfd2125b31b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:20.057250 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529526e-05ca-4b40-9f32-dfd2125b31b9 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:20.057872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529526e-05ca-4b40-9f32-dfd2125b31b9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:20.058173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529526e-05ca-4b40-9f32-dfd2125b31b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:20.059556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529526e-05ca-4b40-9f32-dfd2125b31b9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:20.064864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 439/886] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:09:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:20.070758 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53498ddf-35e9-404e-b312-857bc4c7627f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:20.125102 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:20.174781 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53498ddf-35e9-404e-b312-857bc4c7627f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:20.210993 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c1a9f-1823-45a9-8fca-871c783cf831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:20.211670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c1a9f-1823-45a9-8fca-871c783cf831 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:20.211953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c1a9f-1823-45a9-8fca-871c783cf831 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:20.212228 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c1a9f-1823-45a9-8fca-871c783cf831 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:20.212955 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c1a9f-1823-45a9-8fca-871c783cf831 None admin] RBAC: Adding query filter params (interface=public, service_id=63d7001208774a7697218f47314b88dc, region_id=RegionOne) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:20.213265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c1a9f-1823-45a9-8fca-871c783cf831 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:20.214689 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c1a9f-1823-45a9-8fca-871c783cf831 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:20.223741 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 448/887] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Oct 6 12:09:20 2022] GET /identity/v3/endpoints?service_id=63d7001208774a7697218f47314b88dc&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:20.230346 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529526e-05ca-4b40-9f32-dfd2125b31b9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:20.329266 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc199a15-7004-4fc2-91cc-819834a8a09f None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:20.329459 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc199a15-7004-4fc2-91cc-819834a8a09f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:20.329741 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc199a15-7004-4fc2-91cc-819834a8a09f None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:20.332226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 449/888] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:20.336734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed7260-86af-4bcf-89e7-ccb098022e8a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:20.337273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed7260-86af-4bcf-89e7-ccb098022e8a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:20.337740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed7260-86af-4bcf-89e7-ccb098022e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:20.353711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529526e-05ca-4b40-9f32-dfd2125b31b9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:20.357964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-0aed7260-86af-4bcf-89e7-ccb098022e8a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:20.359404 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0aed7260-86af-4bcf-89e7-ccb098022e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:20.388904 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf8682-d6fb-4773-a27c-02ea41204e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:20.389518 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf8682-d6fb-4773-a27c-02ea41204e9a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:20.389798 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf8682-d6fb-4773-a27c-02ea41204e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:20.390069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf8682-d6fb-4773-a27c-02ea41204e9a None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:20.390670 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf8682-d6fb-4773-a27c-02ea41204e9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:20.392090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf8682-d6fb-4773-a27c-02ea41204e9a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:20.398275 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 440/889] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Oct 6 12:09:20 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:09:20.442362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 450/890] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:20.576707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844ba16-4988-46fd-a525-9a4a88882721 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:20.577058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844ba16-4988-46fd-a525-9a4a88882721 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:20.577266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844ba16-4988-46fd-a525-9a4a88882721 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:20.578858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 441/891] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:20.584956 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5f4d3-4ae7-45d2-a408-8972ff8a9183 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:20.585187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5f4d3-4ae7-45d2-a408-8972ff8a9183 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:20.585465 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5f4d3-4ae7-45d2-a408-8972ff8a9183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:20.604136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3cf5f4d3-4ae7-45d2-a408-8972ff8a9183 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:20.605339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3cf5f4d3-4ae7-45d2-a408-8972ff8a9183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:20.651745 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:20.677451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 451/892] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:20.730047 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844ba16-4988-46fd-a525-9a4a88882721 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:20.762654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:20.838743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844ba16-4988-46fd-a525-9a4a88882721 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:20.873674 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758019b0-4f92-4263-acb7-a8c7abfc2fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:20.874307 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758019b0-4f92-4263-acb7-a8c7abfc2fec None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:20.874586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758019b0-4f92-4263-acb7-a8c7abfc2fec None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:20.874856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758019b0-4f92-4263-acb7-a8c7abfc2fec None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:20.875432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758019b0-4f92-4263-acb7-a8c7abfc2fec None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:20.876839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758019b0-4f92-4263-acb7-a8c7abfc2fec None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:20.886231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-758019b0-4f92-4263-acb7-a8c7abfc2fec None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 12:09:20.886231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:20.886231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:20.886231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:20.886231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:20.886231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:20.886231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:20.886231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:20.886231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 12:09:20.886231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:20.886231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:20.886231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:20.886231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:20.886231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:20.886231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:20.886231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:20.887285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:20.888216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:20.888216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:20.888216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:20.888216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:20.888216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:20.888216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:20.888216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:20.888216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:20.888216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:20.888216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:20.888216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:20.888216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:20.888216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 12:09:20.888216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:20.888216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 442/893] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Oct 6 12:09:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:20.893485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5f4d3-4ae7-45d2-a408-8972ff8a9183 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:20.945023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758019b0-4f92-4263-acb7-a8c7abfc2fec None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:21.002997 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5f4d3-4ae7-45d2-a408-8972ff8a9183 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:21.040606 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab200bcc-44bc-4818-9aac-620d65817c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:21.041343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab200bcc-44bc-4818-9aac-620d65817c50 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:21.041628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab200bcc-44bc-4818-9aac-620d65817c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:21.041897 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab200bcc-44bc-4818-9aac-620d65817c50 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:21.042526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab200bcc-44bc-4818-9aac-620d65817c50 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:21.042827 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab200bcc-44bc-4818-9aac-620d65817c50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:21.044313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab200bcc-44bc-4818-9aac-620d65817c50 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:21.053999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 452/894] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Oct 6 12:09:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:21.054801 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:21.056629 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab200bcc-44bc-4818-9aac-620d65817c50 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:21.058462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758019b0-4f92-4263-acb7-a8c7abfc2fec None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:21.096920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8a12b-4cf8-4b77-91fb-8548627ee382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:21.097473 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8a12b-4cf8-4b77-91fb-8548627ee382 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:21.097754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8a12b-4cf8-4b77-91fb-8548627ee382 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:21.098028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8a12b-4cf8-4b77-91fb-8548627ee382 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:21.103534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:21.113594 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8a12b-4cf8-4b77-91fb-8548627ee382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:21.119614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-1bb8a12b-4cf8-4b77-91fb-8548627ee382 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 12:09:21.119614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:21.119614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:21.119614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:21.119614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:21.119614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:21.119614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:21.119614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:21.119614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:21.119614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:21.119614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:21.119614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:21.119614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:21.119614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:21.119614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:21.119614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:21.120773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:21.120773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:21.120773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:21.120773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:21.120773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:21.120773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:21.120773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:21.120773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:21.120773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:21.120773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:21.120773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:21.120773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:21.120773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:21.120773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:21.120773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:21.120773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:21.121837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:21.121837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:21.121837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:21.121837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:21.121837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:21.121837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:21.121837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:21.121837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:21.121837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:21.121837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:21.121837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 12:09:21.121837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:21.124243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 443/895] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Oct 6 12:09:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:21.131639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8a12b-4cf8-4b77-91fb-8548627ee382 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:21.166682 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab200bcc-44bc-4818-9aac-620d65817c50 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:21.208462 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e3823-95fc-47eb-b920-e6f8c9a3fd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:21.210535 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e3823-95fc-47eb-b920-e6f8c9a3fd14 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:21.210973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e3823-95fc-47eb-b920-e6f8c9a3fd14 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:21.211426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e3823-95fc-47eb-b920-e6f8c9a3fd14 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:21.212230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e3823-95fc-47eb-b920-e6f8c9a3fd14 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:21.212788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e3823-95fc-47eb-b920-e6f8c9a3fd14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:21.214397 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e3823-95fc-47eb-b920-e6f8c9a3fd14 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:21.221327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 453/896] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Oct 6 12:09:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:21.229352 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e3823-95fc-47eb-b920-e6f8c9a3fd14 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:21.258503 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8a12b-4cf8-4b77-91fb-8548627ee382 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:21.294829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebaa36-02a3-4f53-b6c8-65c8e37b13a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:21.295755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebaa36-02a3-4f53-b6c8-65c8e37b13a3 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:21.296040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebaa36-02a3-4f53-b6c8-65c8e37b13a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:21.296315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebaa36-02a3-4f53-b6c8-65c8e37b13a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:21.297040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebaa36-02a3-4f53-b6c8-65c8e37b13a3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:21.297348 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebaa36-02a3-4f53-b6c8-65c8e37b13a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:21.298709 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebaa36-02a3-4f53-b6c8-65c8e37b13a3 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:21.314722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 444/897] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Oct 6 12:09:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:21.316428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebaa36-02a3-4f53-b6c8-65c8e37b13a3 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:21.337402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:21.359070 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e3823-95fc-47eb-b920-e6f8c9a3fd14 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:21.361563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:21.399753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137e3f6-216b-4c96-9a72-971ecaa95a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:21.400358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137e3f6-216b-4c96-9a72-971ecaa95a33 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:21.400641 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137e3f6-216b-4c96-9a72-971ecaa95a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:21.400911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137e3f6-216b-4c96-9a72-971ecaa95a33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:21.401605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137e3f6-216b-4c96-9a72-971ecaa95a33 None admin] RBAC: Adding query filter params (interface=public, service_id=95e6e420ecd24c729d9e098a6b3b55d6, region_id=RegionOne) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:21.401913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137e3f6-216b-4c96-9a72-971ecaa95a33 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:21.403273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137e3f6-216b-4c96-9a72-971ecaa95a33 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:21.413535 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 454/898] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Oct 6 12:09:21 2022] GET /identity/v3/endpoints?service_id=95e6e420ecd24c729d9e098a6b3b55d6&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:21.414388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137e3f6-216b-4c96-9a72-971ecaa95a33 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:21.434661 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebaa36-02a3-4f53-b6c8-65c8e37b13a3 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:21.471393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d9efd-11f7-47d0-b018-3f0d56d5c454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:21.472477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d9efd-11f7-47d0-b018-3f0d56d5c454 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:21.472964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d9efd-11f7-47d0-b018-3f0d56d5c454 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:21.473440 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d9efd-11f7-47d0-b018-3f0d56d5c454 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:21.485332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d9efd-11f7-47d0-b018-3f0d56d5c454 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:21.487149 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d9efd-11f7-47d0-b018-3f0d56d5c454 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:21.503035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-c57d9efd-11f7-47d0-b018-3f0d56d5c454 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:21.503035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:21.503035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:21.503035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:21.503035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:21.503035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:21.503035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:21.503035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:21.503035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:21.503035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:21.503035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:21.503035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:21.503035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:21.503035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:21.503035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:21.503035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:21.504070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:21.505091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:21.505091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:21.505091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:21.511913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 445/899] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:21.513314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d9efd-11f7-47d0-b018-3f0d56d5c454 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:21.520910 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137e3f6-216b-4c96-9a72-971ecaa95a33 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:21.560209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1c7e4-ad03-4d34-8c63-15ebb0b80450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:21.560868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1c7e4-ad03-4d34-8c63-15ebb0b80450 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:21.561154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1c7e4-ad03-4d34-8c63-15ebb0b80450 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:21.561431 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1c7e4-ad03-4d34-8c63-15ebb0b80450 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:21.562035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1c7e4-ad03-4d34-8c63-15ebb0b80450 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:21.564026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1c7e4-ad03-4d34-8c63-15ebb0b80450 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:21.575610 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 455/900] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Oct 6 12:09:21 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:09:21.596043 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:21.636031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d9efd-11f7-47d0-b018-3f0d56d5c454 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:21.675367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fec37f-86c8-4642-a7c8-bf2636bfb961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:21.676424 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fec37f-86c8-4642-a7c8-bf2636bfb961 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:21.676912 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fec37f-86c8-4642-a7c8-bf2636bfb961 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:21.677390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fec37f-86c8-4642-a7c8-bf2636bfb961 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:21.678315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fec37f-86c8-4642-a7c8-bf2636bfb961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:21.678826 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fec37f-86c8-4642-a7c8-bf2636bfb961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:21.680499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fec37f-86c8-4642-a7c8-bf2636bfb961 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:21.699651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 446/901] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:21.701705 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1c7e4-ad03-4d34-8c63-15ebb0b80450 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:21.742578 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:21.817754 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1c7e4-ad03-4d34-8c63-15ebb0b80450 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:21.853881 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbabd1b-1a96-498f-8479-4b041a1259b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:21.854514 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbabd1b-1a96-498f-8479-4b041a1259b6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:21.854794 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbabd1b-1a96-498f-8479-4b041a1259b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:21.855064 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbabd1b-1a96-498f-8479-4b041a1259b6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:21.862315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbabd1b-1a96-498f-8479-4b041a1259b6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:21.863743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbabd1b-1a96-498f-8479-4b041a1259b6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:21.876701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-dbbabd1b-1a96-498f-8479-4b041a1259b6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 12:09:21.876701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:21.876701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:21.876701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:21.876701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:21.876701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:21.876701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:21.876701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:21.876701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:21.876701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:21.876701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:21.876701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:21.876701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:09:21.876701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:09:21.876701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:09:21.876701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:21.877722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:21.878991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:21.878991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:21.878991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:09:21.878991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:09:21.878991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:09:21.878991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:09:21.878991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:09:21.878991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:09:21.878991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 12:09:21.878991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:21.878991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 456/902] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Oct 6 12:09:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:21.886445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fec37f-86c8-4642-a7c8-bf2636bfb961 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:21.994863 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fec37f-86c8-4642-a7c8-bf2636bfb961 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:22.023903 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:22.034717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9c480-859b-4ec3-b6e3-fff4a05ae882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:22.035858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9c480-859b-4ec3-b6e3-fff4a05ae882 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:22.036367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9c480-859b-4ec3-b6e3-fff4a05ae882 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:22.036897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9c480-859b-4ec3-b6e3-fff4a05ae882 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:22.037887 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9c480-859b-4ec3-b6e3-fff4a05ae882 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:22.038414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9c480-859b-4ec3-b6e3-fff4a05ae882 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:22.040142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9c480-859b-4ec3-b6e3-fff4a05ae882 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:22.062169 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 447/903] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Oct 6 12:09:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:22.068968 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbabd1b-1a96-498f-8479-4b041a1259b6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:22.154613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9a8fd-96d0-4d8d-b821-149652734952 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:22.155231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9a8fd-96d0-4d8d-b821-149652734952 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:22.155742 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9a8fd-96d0-4d8d-b821-149652734952 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:22.157544 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 448/904] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:09:22.164147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597945cd-4e9a-472d-a7b2-70843c5b5d8c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:22.164961 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597945cd-4e9a-472d-a7b2-70843c5b5d8c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:22.165443 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597945cd-4e9a-472d-a7b2-70843c5b5d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:22.173204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:22.186358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbabd1b-1a96-498f-8479-4b041a1259b6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:22.187576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-597945cd-4e9a-472d-a7b2-70843c5b5d8c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:22.188855 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-597945cd-4e9a-472d-a7b2-70843c5b5d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:22.226465 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80049590-0453-41a8-81ea-ad861aa6342d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:22.227320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80049590-0453-41a8-81ea-ad861aa6342d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:22.227788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80049590-0453-41a8-81ea-ad861aa6342d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:22.228215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80049590-0453-41a8-81ea-ad861aa6342d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:22.281729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 449/905] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:22.311740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80049590-0453-41a8-81ea-ad861aa6342d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:22.313465 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80049590-0453-41a8-81ea-ad861aa6342d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:22.407043 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:22.422487 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 457/906] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:09:22 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:09:22.424864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597945cd-4e9a-472d-a7b2-70843c5b5d8c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:22.532431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597945cd-4e9a-472d-a7b2-70843c5b5d8c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:22.547585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80049590-0453-41a8-81ea-ad861aa6342d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:22.554110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:22.574485 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e23e9-314d-4987-babb-8e7de82e22c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:22.575372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e23e9-314d-4987-babb-8e7de82e22c8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:22.575825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e23e9-314d-4987-babb-8e7de82e22c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:22.576266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e23e9-314d-4987-babb-8e7de82e22c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:22.586855 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e23e9-314d-4987-babb-8e7de82e22c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:22.588333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e23e9-314d-4987-babb-8e7de82e22c8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:22.603878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-b00e23e9-314d-4987-babb-8e7de82e22c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:22.603878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:22.603878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:22.603878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:22.603878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:22.603878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:22.603878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:22.603878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:22.603878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:22.603878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:22.603878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:22.603878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:22.603878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:22.603878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:22.603878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:22.603878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:22.604965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:22.605969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:22.605969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:22.605969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:22.605969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 450/907] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:22.610702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e23e9-314d-4987-babb-8e7de82e22c8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:22.668311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80049590-0453-41a8-81ea-ad861aa6342d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:22.702913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e024fd2-a577-4ff8-9d16-b724cf933706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:22.703529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e024fd2-a577-4ff8-9d16-b724cf933706 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:22.703814 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e024fd2-a577-4ff8-9d16-b724cf933706 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:22.704081 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e024fd2-a577-4ff8-9d16-b724cf933706 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:22.704715 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e024fd2-a577-4ff8-9d16-b724cf933706 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:22.706067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e024fd2-a577-4ff8-9d16-b724cf933706 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:22.720251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e23e9-314d-4987-babb-8e7de82e22c8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:22.723552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 458/908] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Oct 6 12:09:22 2022] POST /identity/v3/services => generated 241 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:09:22.723552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa0fca-0ee4-4362-b0cb-537aea8ec226 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:22.723837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa0fca-0ee4-4362-b0cb-537aea8ec226 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:22.724111 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa0fca-0ee4-4362-b0cb-537aea8ec226 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:22.727546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 459/909] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:22.731183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23204f8b-5583-4d58-9c52-937d44212898 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:22.731722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23204f8b-5583-4d58-9c52-937d44212898 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:22.732157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23204f8b-5583-4d58-9c52-937d44212898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:22.754834 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-23204f8b-5583-4d58-9c52-937d44212898 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:22.756768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-23204f8b-5583-4d58-9c52-937d44212898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:22.776154 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d63a3a-3157-4841-9901-d49ceaf436df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:22.776988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d63a3a-3157-4841-9901-d49ceaf436df None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:22.777269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d63a3a-3157-4841-9901-d49ceaf436df None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:22.777542 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d63a3a-3157-4841-9901-d49ceaf436df None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:22.778194 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d63a3a-3157-4841-9901-d49ceaf436df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:22.778495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d63a3a-3157-4841-9901-d49ceaf436df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:22.780280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d63a3a-3157-4841-9901-d49ceaf436df None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:22.801925 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 451/910] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:22.804470 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d63a3a-3157-4841-9901-d49ceaf436df None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:22.834636 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:22.836915 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:22.868680 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 460/911] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:22.928391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d63a3a-3157-4841-9901-d49ceaf436df None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:22.967829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042d69f-d2b9-47a3-8eef-04b87089c496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:22.968846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042d69f-d2b9-47a3-8eef-04b87089c496 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:22.969343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042d69f-d2b9-47a3-8eef-04b87089c496 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:22.969847 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042d69f-d2b9-47a3-8eef-04b87089c496 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:22.977192 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042d69f-d2b9-47a3-8eef-04b87089c496 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:22.978908 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042d69f-d2b9-47a3-8eef-04b87089c496 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:22.991264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-b042d69f-d2b9-47a3-8eef-04b87089c496 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:22.991264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:22.991264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:22.991264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:22.991264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:22.991264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:22.991264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:22.991264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:22.991264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:22.991264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:22.991264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:22.991264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:22.991264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:09:22.991264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:09:22.991264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:22.992357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:22.993325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:09:22.993325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:09:22.993325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:09:22.993325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:09:22.993325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:09:22.993325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:09:22.993325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:22.993325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:22.999601 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 452/912] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:23.001796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23204f8b-5583-4d58-9c52-937d44212898 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:23.102336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23204f8b-5583-4d58-9c52-937d44212898 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:23.137029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb239f39-6e9f-4f79-9f09-c454be9937fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:23.137993 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb239f39-6e9f-4f79-9f09-c454be9937fd None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:23.138491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb239f39-6e9f-4f79-9f09-c454be9937fd None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:23.138972 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb239f39-6e9f-4f79-9f09-c454be9937fd None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:23.139913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb239f39-6e9f-4f79-9f09-c454be9937fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:23.140547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb239f39-6e9f-4f79-9f09-c454be9937fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:23.142316 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb239f39-6e9f-4f79-9f09-c454be9937fd None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:23.146305 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042d69f-d2b9-47a3-8eef-04b87089c496 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:23.159620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 461/913] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:09:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:23.165762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb239f39-6e9f-4f79-9f09-c454be9937fd None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:23.216050 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:23.263957 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:23.286500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042d69f-d2b9-47a3-8eef-04b87089c496 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:23.290798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb239f39-6e9f-4f79-9f09-c454be9937fd None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:23.334594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03186f8-ce1d-4758-ae96-b8b5774565ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:23.335227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03186f8-ce1d-4758-ae96-b8b5774565ac None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:23.335538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03186f8-ce1d-4758-ae96-b8b5774565ac None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:23.335801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03186f8-ce1d-4758-ae96-b8b5774565ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:23.336524 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03186f8-ce1d-4758-ae96-b8b5774565ac None admin] RBAC: Adding query filter params (role.id=14d78f802e8d43d684535749f98404c7, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=96a302378dc0416d87c1cf11a3ba0a9c) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:23.336830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03186f8-ce1d-4758-ae96-b8b5774565ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:23.338261 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03186f8-ce1d-4758-ae96-b8b5774565ac None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:23.340486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c231cf-80b7-4695-af9a-1d57bbe60a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:23.341974 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c231cf-80b7-4695-af9a-1d57bbe60a33 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:23.342740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c231cf-80b7-4695-af9a-1d57bbe60a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:23.343233 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c231cf-80b7-4695-af9a-1d57bbe60a33 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:23.344053 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c231cf-80b7-4695-af9a-1d57bbe60a33 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:23.344416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 462/914] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:09:23 2022] GET /identity/v3/role_assignments?user.id=96a302378dc0416d87c1cf11a3ba0a9c&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=14d78f802e8d43d684535749f98404c7 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:23.345815 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c231cf-80b7-4695-af9a-1d57bbe60a33 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-72c231cf-80b7-4695-af9a-1d57bbe60a33 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:23.359938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:23.361091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:23.361091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:23.361091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:23.361091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:23.361091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:23.361091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:23.361091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:23.361091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:23.361091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:23.361091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:23.361091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:23.361091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:23.361091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:23.361091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:23.361091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:23.361091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:23.361984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:23.361984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:23.361984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:23.361984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:23.361984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:23.361984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:23.361984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:23.361984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:23.361984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:23.361984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:23.361984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:23.361984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:23.361984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 12:09:23.361984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:23.365079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 453/915] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:09:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:23.371355 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03186f8-ce1d-4758-ae96-b8b5774565ac None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:23.485984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03186f8-ce1d-4758-ae96-b8b5774565ac None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:23.495163 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:23.540196 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb7d52-d7ea-4345-89d3-06010234cdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:23.541775 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb7d52-d7ea-4345-89d3-06010234cdfb None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:23.542306 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb7d52-d7ea-4345-89d3-06010234cdfb None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:23.542590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb7d52-d7ea-4345-89d3-06010234cdfb None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:23.543281 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb7d52-d7ea-4345-89d3-06010234cdfb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:23.543624 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb7d52-d7ea-4345-89d3-06010234cdfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:23.545013 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb7d52-d7ea-4345-89d3-06010234cdfb None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:23.550763 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 463/916] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:09:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:23.561895 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c231cf-80b7-4695-af9a-1d57bbe60a33 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:23.760902 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c231cf-80b7-4695-af9a-1d57bbe60a33 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:23.807201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9039de-f775-4fd6-b0b0-960b3cc4a248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:23.808208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9039de-f775-4fd6-b0b0-960b3cc4a248 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:23.808723 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9039de-f775-4fd6-b0b0-960b3cc4a248 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:23.809194 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9039de-f775-4fd6-b0b0-960b3cc4a248 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:23.810147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9039de-f775-4fd6-b0b0-960b3cc4a248 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:23.811126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9039de-f775-4fd6-b0b0-960b3cc4a248 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:23.812935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9039de-f775-4fd6-b0b0-960b3cc4a248 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:23.822435 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 454/917] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Oct 6 12:09:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:23.828396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb7d52-d7ea-4345-89d3-06010234cdfb None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:24.025023 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb7d52-d7ea-4345-89d3-06010234cdfb None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:24.027698 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6850b64-3056-47f8-8e8f-9d37ecc68e9c None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:24.027986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6850b64-3056-47f8-8e8f-9d37ecc68e9c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:24.028263 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6850b64-3056-47f8-8e8f-9d37ecc68e9c None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:24.029974 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 455/918] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:09:24.034844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf84f-eb1b-4519-8c5d-90ebee62c0f4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:24.035127 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf84f-eb1b-4519-8c5d-90ebee62c0f4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:24.035398 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf84f-eb1b-4519-8c5d-90ebee62c0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:24.068748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e3bcf84f-eb1b-4519-8c5d-90ebee62c0f4 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:24.069939 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e3bcf84f-eb1b-4519-8c5d-90ebee62c0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:24.083994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d35bfa-916f-4752-ade8-4335073d5b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:24.084954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d35bfa-916f-4752-ade8-4335073d5b67 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:24.085455 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d35bfa-916f-4752-ade8-4335073d5b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:24.086718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d35bfa-916f-4752-ade8-4335073d5b67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:24.089105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d35bfa-916f-4752-ade8-4335073d5b67 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:24.091911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d35bfa-916f-4752-ade8-4335073d5b67 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:24.095487 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:24.133008 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 464/919] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:09:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:09:24.196597 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 456/920] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:24.464061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d35bfa-916f-4752-ade8-4335073d5b67 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:24.493583 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:24.571483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d35bfa-916f-4752-ade8-4335073d5b67 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:24.606296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ae80a-807d-46b2-b3ad-16490f6f17f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:24.607204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ae80a-807d-46b2-b3ad-16490f6f17f5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:24.607674 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ae80a-807d-46b2-b3ad-16490f6f17f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:24.608093 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ae80a-807d-46b2-b3ad-16490f6f17f5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:24.608829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ae80a-807d-46b2-b3ad-16490f6f17f5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:24.610365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ae80a-807d-46b2-b3ad-16490f6f17f5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:24.619870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-a19ae80a-807d-46b2-b3ad-16490f6f17f5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 12:09:24.619870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:24.619870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:24.619870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:24.619870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:24.619870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:24.619870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:24.619870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:24.619870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 12:09:24.619870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:24.619870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:24.619870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:24.619870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:24.619870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:24.619870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:24.619870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:24.621044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:24.622180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:24.622180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:24.622180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:24.622180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:24.622180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:24.622180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:24.622180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:24.622180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:24.622180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:24.622180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:24.622180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:24.622180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:24.622180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 12:09:24.622180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:24.624694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 465/921] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Oct 6 12:09:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:24.632705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf84f-eb1b-4519-8c5d-90ebee62c0f4 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:24.737432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf84f-eb1b-4519-8c5d-90ebee62c0f4 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:24.773617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d10819-3794-401f-aea7-e00ca8e43b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:24.774497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d10819-3794-401f-aea7-e00ca8e43b55 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:24.774940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d10819-3794-401f-aea7-e00ca8e43b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:24.775359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d10819-3794-401f-aea7-e00ca8e43b55 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:24.776154 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d10819-3794-401f-aea7-e00ca8e43b55 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:24.776612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d10819-3794-401f-aea7-e00ca8e43b55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:24.778133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d10819-3794-401f-aea7-e00ca8e43b55 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:24.784319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 457/922] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Oct 6 12:09:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:24.792787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ae80a-807d-46b2-b3ad-16490f6f17f5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:24.886316 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:24.896820 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ae80a-807d-46b2-b3ad-16490f6f17f5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:24.933104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a31cf8-9921-4eb8-b6e1-bbf60fac1631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:24.933737 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a31cf8-9921-4eb8-b6e1-bbf60fac1631 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:24.934009 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a31cf8-9921-4eb8-b6e1-bbf60fac1631 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:24.934289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a31cf8-9921-4eb8-b6e1-bbf60fac1631 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:24.934942 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a31cf8-9921-4eb8-b6e1-bbf60fac1631 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:24.935249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a31cf8-9921-4eb8-b6e1-bbf60fac1631 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:24.936703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a31cf8-9921-4eb8-b6e1-bbf60fac1631 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:24.943388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 466/923] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Oct 6 12:09:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:24.952717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d10819-3794-401f-aea7-e00ca8e43b55 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:24.956555 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:25.058416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d10819-3794-401f-aea7-e00ca8e43b55 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:25.069785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99e81d-274b-44b8-8fd1-573554092f55 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:25.070073 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99e81d-274b-44b8-8fd1-573554092f55 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:25.070352 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99e81d-274b-44b8-8fd1-573554092f55 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:25.071961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 467/924] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:09:25.076852 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c53d85-1f00-415e-ab87-5c81d56f1179 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:25.077152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c53d85-1f00-415e-ab87-5c81d56f1179 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:25.077427 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c53d85-1f00-415e-ab87-5c81d56f1179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:25.094391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25154048-6315-4f91-92a6-8a9296705e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:25.095297 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25154048-6315-4f91-92a6-8a9296705e44 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:25.095775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25154048-6315-4f91-92a6-8a9296705e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:25.096275 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25154048-6315-4f91-92a6-8a9296705e44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:25.096984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d9c53d85-1f00-415e-ab87-5c81d56f1179 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:25.097324 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25154048-6315-4f91-92a6-8a9296705e44 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:25.098183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d9c53d85-1f00-415e-ab87-5c81d56f1179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:25.099062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25154048-6315-4f91-92a6-8a9296705e44 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:25.126290 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 458/925] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:09:24 2022] POST /identity/v3/endpoints => generated 347 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:09:25.186860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 468/926] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:25.454896 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25154048-6315-4f91-92a6-8a9296705e44 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:25.520980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:25.566107 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25154048-6315-4f91-92a6-8a9296705e44 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:25.600931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c1cf6-8ce0-421e-b01f-d08d6d9f7b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:25.601568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c1cf6-8ce0-421e-b01f-d08d6d9f7b79 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:25.601845 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c1cf6-8ce0-421e-b01f-d08d6d9f7b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:25.602113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c1cf6-8ce0-421e-b01f-d08d6d9f7b79 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:25.602690 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c1cf6-8ce0-421e-b01f-d08d6d9f7b79 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:25.604144 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c1cf6-8ce0-421e-b01f-d08d6d9f7b79 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-278c1cf6-8ce0-421e-b01f-d08d6d9f7b79 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:25.613020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:25.614062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:25.614062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:25.614062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:25.614062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:25.614062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:25.614062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:25.614062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:25.614062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:25.614062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:25.614062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:25.614062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:25.614062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:25.614062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:25.614062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:25.614062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:25.614062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:25.615017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:25.615017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:25.615017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:25.615017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:25.615017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:25.615017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:25.615017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:25.615017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:25.615017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:25.615017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:25.615017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:25.615017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:25.615017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 12:09:25.615017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:25.615017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 459/927] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Oct 6 12:09:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:25.618842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872c10-b20c-43f5-b6c7-09acc0d47a62 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:25.619385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872c10-b20c-43f5-b6c7-09acc0d47a62 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:25.619385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08872c10-b20c-43f5-b6c7-09acc0d47a62 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:25.624891 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 469/928] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:25.625581 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c1cf6-8ce0-421e-b01f-d08d6d9f7b79 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:25.626002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f8568-90f0-46ec-91b6-67618421cecf None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:25.626240 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f8568-90f0-46ec-91b6-67618421cecf None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:25.626518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f8568-90f0-46ec-91b6-67618421cecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:25.645718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-9b6f8568-90f0-46ec-91b6-67618421cecf None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:25.647084 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9b6f8568-90f0-46ec-91b6-67618421cecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:25.735739 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 470/929] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:25.749423 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c1cf6-8ce0-421e-b01f-d08d6d9f7b79 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:25.773834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:25.786004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51583878-7794-4f9d-8a1a-59860016274b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:25.786638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51583878-7794-4f9d-8a1a-59860016274b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:25.786950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51583878-7794-4f9d-8a1a-59860016274b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:25.787225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51583878-7794-4f9d-8a1a-59860016274b None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:25.787880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51583878-7794-4f9d-8a1a-59860016274b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:25.788186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51583878-7794-4f9d-8a1a-59860016274b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:25.789797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51583878-7794-4f9d-8a1a-59860016274b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:25.794760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 460/930] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Oct 6 12:09:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:25.804595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f8568-90f0-46ec-91b6-67618421cecf None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:25.816976 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:25.914465 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f8568-90f0-46ec-91b6-67618421cecf None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:25.950323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d00654-c86f-4bac-b798-2998caf42c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:25.950973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d00654-c86f-4bac-b798-2998caf42c50 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:25.951221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d00654-c86f-4bac-b798-2998caf42c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:25.951525 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d00654-c86f-4bac-b798-2998caf42c50 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:25.952160 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d00654-c86f-4bac-b798-2998caf42c50 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:25.952465 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d00654-c86f-4bac-b798-2998caf42c50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:25.953842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d00654-c86f-4bac-b798-2998caf42c50 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:25.959702 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 471/931] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Oct 6 12:09:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:25.968860 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51583878-7794-4f9d-8a1a-59860016274b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:26.048056 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d00654-c86f-4bac-b798-2998caf42c50 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:26.081290 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51583878-7794-4f9d-8a1a-59860016274b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:26.117908 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2a3cf-4eb6-41d7-99f6-ad2bdb33ae66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:26.118537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2a3cf-4eb6-41d7-99f6-ad2bdb33ae66 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:26.118813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2a3cf-4eb6-41d7-99f6-ad2bdb33ae66 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:26.119366 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2a3cf-4eb6-41d7-99f6-ad2bdb33ae66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:26.119796 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2a3cf-4eb6-41d7-99f6-ad2bdb33ae66 None admin] RBAC: Adding query filter params (interface=public, service_id=8df23e93489d4b119af080728e844df6, region_id=RegionOne) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:26.120101 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2a3cf-4eb6-41d7-99f6-ad2bdb33ae66 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:26.121470 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2a3cf-4eb6-41d7-99f6-ad2bdb33ae66 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:26.127288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 461/932] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Oct 6 12:09:25 2022] GET /identity/v3/endpoints?service_id=8df23e93489d4b119af080728e844df6&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:26.133347 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2a3cf-4eb6-41d7-99f6-ad2bdb33ae66 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:26.142216 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:26.170767 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d00654-c86f-4bac-b798-2998caf42c50 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:26.185789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:26.207871 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d668174-3c62-44c2-9d36-2a9c1b345cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:26.208475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d668174-3c62-44c2-9d36-2a9c1b345cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:26.208788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d668174-3c62-44c2-9d36-2a9c1b345cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:26.209069 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d668174-3c62-44c2-9d36-2a9c1b345cc8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:26.219604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d668174-3c62-44c2-9d36-2a9c1b345cc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:26.225054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-1d668174-3c62-44c2-9d36-2a9c1b345cc8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 12:09:26.225054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:26.225054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:26.225054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:26.225054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:26.225054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:26.225054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:26.225054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:26.225054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:26.225054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:26.225054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:26.225054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:26.225054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:26.225054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:26.225054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:26.225054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:26.225999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:26.225999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:26.225999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:26.225999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:26.225999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:26.225999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:26.225999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:26.225999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:26.225999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:26.225999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:26.225999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:26.225999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:26.225999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:26.225999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:26.225999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:26.225999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:26.226862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:26.226862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:26.226862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:26.226862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:26.226862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:26.226862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:26.226862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:26.226862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:26.226862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:26.226862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:26.226862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 12:09:26.226862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:26.226862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 472/933] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Oct 6 12:09:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:26.232443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d668174-3c62-44c2-9d36-2a9c1b345cc8 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:26.251681 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2a3cf-4eb6-41d7-99f6-ad2bdb33ae66 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:26.287626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb75e03-69f4-4e29-b730-eb4ae0944017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:26.288259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb75e03-69f4-4e29-b730-eb4ae0944017 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:26.288591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb75e03-69f4-4e29-b730-eb4ae0944017 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:26.289040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb75e03-69f4-4e29-b730-eb4ae0944017 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:26.289689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb75e03-69f4-4e29-b730-eb4ae0944017 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:26.291049 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb75e03-69f4-4e29-b730-eb4ae0944017 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:26.297488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 462/934] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Oct 6 12:09:26 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:09:26.371309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d668174-3c62-44c2-9d36-2a9c1b345cc8 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:26.412219 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa3a29-365c-47dc-8fea-3f9af9ab8ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:26.413211 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa3a29-365c-47dc-8fea-3f9af9ab8ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:26.413699 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa3a29-365c-47dc-8fea-3f9af9ab8ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:26.414175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa3a29-365c-47dc-8fea-3f9af9ab8ed7 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:26.415084 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa3a29-365c-47dc-8fea-3f9af9ab8ed7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:26.415665 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa3a29-365c-47dc-8fea-3f9af9ab8ed7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:26.417280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa3a29-365c-47dc-8fea-3f9af9ab8ed7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:26.419136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:26.431680 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 473/935] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Oct 6 12:09:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:26.434701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb75e03-69f4-4e29-b730-eb4ae0944017 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:26.459426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284fa8b-265a-4082-a4eb-53edbd9c71ac None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:26.459793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284fa8b-265a-4082-a4eb-53edbd9c71ac None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:26.460018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284fa8b-265a-4082-a4eb-53edbd9c71ac None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:26.462190 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 474/936] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:26.467321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480d4e7-8885-4706-9150-1f93f1d5db2b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:26.467645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480d4e7-8885-4706-9150-1f93f1d5db2b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:26.467868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480d4e7-8885-4706-9150-1f93f1d5db2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:26.488083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8480d4e7-8885-4706-9150-1f93f1d5db2b None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:26.489285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8480d4e7-8885-4706-9150-1f93f1d5db2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:26.558336 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb75e03-69f4-4e29-b730-eb4ae0944017 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:26.567780 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 475/937] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:26.575350 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:26.597127 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296f60e-0105-4b71-b4e0-964dec323cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:26.597990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296f60e-0105-4b71-b4e0-964dec323cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:26.598421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296f60e-0105-4b71-b4e0-964dec323cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:26.598870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296f60e-0105-4b71-b4e0-964dec323cb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:26.609882 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296f60e-0105-4b71-b4e0-964dec323cb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:26.611489 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296f60e-0105-4b71-b4e0-964dec323cb5 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:26.626572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-a296f60e-0105-4b71-b4e0-964dec323cb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:26.626572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:26.626572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:26.626572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:26.626572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:26.626572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:26.626572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:26.626572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:26.626572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:26.626572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:26.626572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:26.626572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:26.626572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:26.626572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:26.626572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:26.626572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:26.627779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:26.628926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:26.628926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:26.628926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:26.635090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 463/938] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:26.636662 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480d4e7-8885-4706-9150-1f93f1d5db2b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:26.742831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480d4e7-8885-4706-9150-1f93f1d5db2b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:26.770328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:26.781348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4ef45-1a67-41ff-bea8-68494fa4abf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:26.782213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4ef45-1a67-41ff-bea8-68494fa4abf3 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:26.782676 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4ef45-1a67-41ff-bea8-68494fa4abf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:26.783141 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4ef45-1a67-41ff-bea8-68494fa4abf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:26.784080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4ef45-1a67-41ff-bea8-68494fa4abf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:26.784581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4ef45-1a67-41ff-bea8-68494fa4abf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:26.786140 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4ef45-1a67-41ff-bea8-68494fa4abf3 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:26.811593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 476/939] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:26.813389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296f60e-0105-4b71-b4e0-964dec323cb5 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:26.854190 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4ef45-1a67-41ff-bea8-68494fa4abf3 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:26.934764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296f60e-0105-4b71-b4e0-964dec323cb5 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:26.974752 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4ef45-1a67-41ff-bea8-68494fa4abf3 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:26.976606 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19d75e-35b1-4c03-bf4d-cbe7ba3ee0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:26.977578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19d75e-35b1-4c03-bf4d-cbe7ba3ee0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:26.978058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19d75e-35b1-4c03-bf4d-cbe7ba3ee0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:26.978560 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19d75e-35b1-4c03-bf4d-cbe7ba3ee0a5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:26.987135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19d75e-35b1-4c03-bf4d-cbe7ba3ee0a5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:26.989187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19d75e-35b1-4c03-bf4d-cbe7ba3ee0a5 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:27.001429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-5d19d75e-35b1-4c03-bf4d-cbe7ba3ee0a5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 12:09:27.001429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:27.001429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:27.001429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:27.001429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:27.001429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:27.001429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:27.001429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:27.001429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:27.001429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:27.001429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:27.001429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:27.001429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:09:27.001429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:09:27.001429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:09:27.001429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:27.002439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:27.003427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:27.003427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:27.003427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:09:27.003427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:09:27.003427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:09:27.003427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:09:27.003427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:09:27.003427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:09:27.003427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 12:09:27.003427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:27.006870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 464/940] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Oct 6 12:09:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:27.012777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19d75e-35b1-4c03-bf4d-cbe7ba3ee0a5 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:27.015870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8e2d0-6d58-423c-8659-7bd6b0fb3e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:27.017697 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8e2d0-6d58-423c-8659-7bd6b0fb3e8e None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:27.018206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8e2d0-6d58-423c-8659-7bd6b0fb3e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:27.018678 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8e2d0-6d58-423c-8659-7bd6b0fb3e8e None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:27.019594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8e2d0-6d58-423c-8659-7bd6b0fb3e8e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:27.021809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8e2d0-6d58-423c-8659-7bd6b0fb3e8e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:27.025797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: WARNING keystone.server.flask.common [None req-2cf8e2d0-6d58-423c-8659-7bd6b0fb3e8e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 06 12:09:27.045003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:27.051906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.sql.core [None req-2cf8e2d0-6d58-423c-8659-7bd6b0fb3e8e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 06 12:09:27.051906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 06 12:09:27.051906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [parameters: {'id': 'ae3ca764e3d14bff9620e2e7ff5c09a1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 06 12:09:27.051906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81355) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 06 12:09:27.076213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-2cf8e2d0-6d58-423c-8659-7bd6b0fb3e8e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ae3ca764e3d14bff9620e2e7ff5c09a1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ae3ca764e3d14bff9620e2e7ff5c09a1. Oct 06 12:09:27.076213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:27.076213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 12:09:27.076213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 12:09:27.076213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 12:09:27.076213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 12:09:27.076213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 12:09:27.076213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 12:09:27.076213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 12:09:27.076213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application conn.query(q) Oct 06 12:09:27.076213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 12:09:27.076213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 12:09:27.076213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application result.read() Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 06 12:09:27.077101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application next(self.gen) Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application yield resource Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application next(self.gen) Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self.session.rollback() Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self.force_reraise() Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise self.value Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application session.commit() Oct 06 12:09:27.078089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._prepare_impl() Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self.session.flush() Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._flush(objects) Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application compat.raise_( Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application flush_context.execute() Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 06 12:09:27.079062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rec.execute(self) Oct 06 12:09:27.080016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 06 12:09:27.080016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 06 12:09:27.080016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 06 12:09:27.080016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 06 12:09:27.080016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 06 12:09:27.080016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 06 12:09:27.080016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 06 12:09:27.080016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 06 12:09:27.080016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 06 12:09:27.080016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 06 12:09:27.080016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 06 12:09:27.080016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 06 12:09:27.080016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 06 12:09:27.080016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 06 12:09:27.080016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application conn.query(q) Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application result.read() Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 12:09:27.080860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 12:09:27.081746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 12:09:27.081746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 12:09:27.081746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 12:09:27.081746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 12:09:27.081746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 12:09:27.081746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 12:09:27.081746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 06 12:09:27.081746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 06 12:09:27.081746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [parameters: {'id': 'ae3ca764e3d14bff9620e2e7ff5c09a1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 06 12:09:27.081746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 06 12:09:27.081746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:27.081746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:27.081746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:27.081746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:27.081746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:27.082558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:27.088948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:27.088948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 06 12:09:27.088948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 06 12:09:27.088948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:27.088948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:27.088948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 06 12:09:27.088948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 06 12:09:27.088948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ae3ca764e3d14bff9620e2e7ff5c09a1. Oct 06 12:09:27.088948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:27.088948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 477/941] 10.222.0.33 () {60 vars in 1140 bytes} [Thu Oct 6 12:09:26 2022] POST /identity/v3/projects => generated 262 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:27.090374 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8e2d0-6d58-423c-8659-7bd6b0fb3e8e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:27.135290 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19d75e-35b1-4c03-bf4d-cbe7ba3ee0a5 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:27.144876 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:27.173881 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9791700-6ad5-48fb-95fd-8dfaf4137a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:27.174570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9791700-6ad5-48fb-95fd-8dfaf4137a7f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:27.174856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9791700-6ad5-48fb-95fd-8dfaf4137a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:27.175132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9791700-6ad5-48fb-95fd-8dfaf4137a7f None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:27.175934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9791700-6ad5-48fb-95fd-8dfaf4137a7f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:27.176357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9791700-6ad5-48fb-95fd-8dfaf4137a7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:27.178035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9791700-6ad5-48fb-95fd-8dfaf4137a7f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:27.201924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 465/942] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Oct 6 12:09:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:27.203433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9791700-6ad5-48fb-95fd-8dfaf4137a7f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:27.218078 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8e2d0-6d58-423c-8659-7bd6b0fb3e8e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:27.257669 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada5f8e-42e7-40a1-8862-40e2d8fd8850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:27.258444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada5f8e-42e7-40a1-8862-40e2d8fd8850 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:27.258818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada5f8e-42e7-40a1-8862-40e2d8fd8850 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:27.259203 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada5f8e-42e7-40a1-8862-40e2d8fd8850 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:27.268441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada5f8e-42e7-40a1-8862-40e2d8fd8850 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:27.271598 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada5f8e-42e7-40a1-8862-40e2d8fd8850 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:27.285075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-aada5f8e-42e7-40a1-8862-40e2d8fd8850 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:27.285075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:27.285075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:27.285075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:27.285075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:27.285075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:27.285075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:27.285075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:27.285075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:27.285075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:27.285075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:27.285075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:27.285075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:09:27.285075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:09:27.285075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:27.286221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:27.287421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:09:27.287421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:09:27.287421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:09:27.287421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:09:27.287421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:09:27.287421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:09:27.287421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:27.287421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:27.291735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 478/943] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:27.295851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada5f8e-42e7-40a1-8862-40e2d8fd8850 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:27.326100 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:27.347801 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9791700-6ad5-48fb-95fd-8dfaf4137a7f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:27.385507 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ab37d-f232-4caf-8ea5-02fb17c7a3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:27.386400 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ab37d-f232-4caf-8ea5-02fb17c7a3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:27.386851 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ab37d-f232-4caf-8ea5-02fb17c7a3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:27.387283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ab37d-f232-4caf-8ea5-02fb17c7a3d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:27.398942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ab37d-f232-4caf-8ea5-02fb17c7a3d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:27.400687 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ab37d-f232-4caf-8ea5-02fb17c7a3d2 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:27.417419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-b13ab37d-f232-4caf-8ea5-02fb17c7a3d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:27.417419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:27.417419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:27.417419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:27.417419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:27.417419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:27.417419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:27.417419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:27.417419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:27.417419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:27.417419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:27.417419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:27.417419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:27.417419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:27.417419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:27.417419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:27.418669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:27.419832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:27.419832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:27.419832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:27.420021 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada5f8e-42e7-40a1-8862-40e2d8fd8850 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:27.420021 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:27.420726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 466/944] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:27.422046 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e74d16-a013-4cda-9526-f1862bc03b18 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:27.422316 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e74d16-a013-4cda-9526-f1862bc03b18 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:27.422627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e74d16-a013-4cda-9526-f1862bc03b18 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:27.424929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 467/945] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:27.428378 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e74d16-a013-4cda-9526-f1862bc03b18 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:27.462574 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffaa8ec-bbdf-4501-8ed1-2f404bae90a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:27.463107 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaa8ec-bbdf-4501-8ed1-2f404bae90a5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:27.463385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaa8ec-bbdf-4501-8ed1-2f404bae90a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:27.463679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaa8ec-bbdf-4501-8ed1-2f404bae90a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:27.464335 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffaa8ec-bbdf-4501-8ed1-2f404bae90a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:27.464637 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffaa8ec-bbdf-4501-8ed1-2f404bae90a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:27.466059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffaa8ec-bbdf-4501-8ed1-2f404bae90a5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:27.495536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 479/946] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Oct 6 12:09:27 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:27.495536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b31685-2d78-4d51-a8ac-b137ec06c76a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:27.495536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b31685-2d78-4d51-a8ac-b137ec06c76a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:27.495536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b31685-2d78-4d51-a8ac-b137ec06c76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:27.513023 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c5b31685-2d78-4d51-a8ac-b137ec06c76a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:27.515610 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c5b31685-2d78-4d51-a8ac-b137ec06c76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:27.555533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e74d16-a013-4cda-9526-f1862bc03b18 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:27.598561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84de91-701e-4442-aaeb-bece36a13365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:27.599786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84de91-701e-4442-aaeb-bece36a13365 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:27.603561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84de91-701e-4442-aaeb-bece36a13365 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:27.603561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84de91-701e-4442-aaeb-bece36a13365 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:27.603561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84de91-701e-4442-aaeb-bece36a13365 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:27.603561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84de91-701e-4442-aaeb-bece36a13365 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:27.603561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84de91-701e-4442-aaeb-bece36a13365 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:27.606699 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:27.608596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 480/947] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:27.624083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 468/948] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:27.630804 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b31685-2d78-4d51-a8ac-b137ec06c76a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:27.730690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:27.741078 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b31685-2d78-4d51-a8ac-b137ec06c76a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:27.776030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c5b02-37ba-449a-94c2-f5547c96ad71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:27.776655 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c5b02-37ba-449a-94c2-f5547c96ad71 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:27.776931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c5b02-37ba-449a-94c2-f5547c96ad71 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:27.777200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c5b02-37ba-449a-94c2-f5547c96ad71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:27.784201 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c5b02-37ba-449a-94c2-f5547c96ad71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:27.785623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c5b02-37ba-449a-94c2-f5547c96ad71 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:27.797436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-2b2c5b02-37ba-449a-94c2-f5547c96ad71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:27.797436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:27.797436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:27.797436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:27.797436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:27.797436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:27.797436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:27.797436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:27.797436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:27.797436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:27.797436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:27.797436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:27.797436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:09:27.797436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:09:27.797436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:27.798350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:27.799265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:09:27.799265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:09:27.799265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:09:27.799265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:09:27.799265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:09:27.799265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:09:27.799265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:27.799265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:27.799265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 481/949] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:27.804689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84de91-701e-4442-aaeb-bece36a13365 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:27.886802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c5b02-37ba-449a-94c2-f5547c96ad71 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:27.912911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84de91-701e-4442-aaeb-bece36a13365 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:27.947685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e7485-9aef-467f-a109-1f7f67663633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:27.948255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e7485-9aef-467f-a109-1f7f67663633 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:27.948538 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e7485-9aef-467f-a109-1f7f67663633 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:27.948811 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e7485-9aef-467f-a109-1f7f67663633 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:27.949508 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e7485-9aef-467f-a109-1f7f67663633 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:27.949815 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e7485-9aef-467f-a109-1f7f67663633 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:27.951254 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e7485-9aef-467f-a109-1f7f67663633 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:27.964606 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 469/950] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:09:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:27.970145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e7485-9aef-467f-a109-1f7f67663633 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:28.003610 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c5b02-37ba-449a-94c2-f5547c96ad71 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:28.044739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:28.055396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c3af8-bccc-4570-b46a-01bb44de60c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:28.056081 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c3af8-bccc-4570-b46a-01bb44de60c0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:28.056367 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c3af8-bccc-4570-b46a-01bb44de60c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:28.056645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c3af8-bccc-4570-b46a-01bb44de60c0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:28.057241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c3af8-bccc-4570-b46a-01bb44de60c0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:28.058674 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c3af8-bccc-4570-b46a-01bb44de60c0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-0b3c3af8-bccc-4570-b46a-01bb44de60c0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:28.070385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:28.071444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:28.071444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:28.071444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:28.071444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:28.071444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:28.071444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:28.071444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:28.071444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:28.071444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:28.071444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:28.071444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:28.071444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:28.071444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:28.071444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:28.071444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:28.071444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:28.072631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:28.072631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:28.072631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:28.072631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:28.072631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:28.072631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:28.072631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:28.072631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:28.072631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:28.072631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:28.072631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:28.072631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:28.072631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 12:09:28.072631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:28.072631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 482/951] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:28.081556 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c3af8-bccc-4570-b46a-01bb44de60c0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:28.107132 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:28.110584 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e7485-9aef-467f-a109-1f7f67663633 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:28.159239 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76308a09-de24-430e-92fa-e2d58c77ffb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:28.160468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76308a09-de24-430e-92fa-e2d58c77ffb5 None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:28.160981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76308a09-de24-430e-92fa-e2d58c77ffb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:28.161462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76308a09-de24-430e-92fa-e2d58c77ffb5 None admin] PATH_INFO: `/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/96a302378dc0416d87c1cf11a3ba0a9c/roles/14d78f802e8d43d684535749f98404c7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:28.164065 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76308a09-de24-430e-92fa-e2d58c77ffb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=314d3d13ba2e4a0f93f5d3f34c50a96f, user_id=96a302378dc0416d87c1cf11a3ba0a9c, role_id=14d78f802e8d43d684535749f98404c7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:28.165776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76308a09-de24-430e-92fa-e2d58c77ffb5 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:28.184511 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 470/952] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:09:27 2022] PUT /identity/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/96a302378dc0416d87c1cf11a3ba0a9c/roles/14d78f802e8d43d684535749f98404c7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:09:28.214215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c3af8-bccc-4570-b46a-01bb44de60c0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:28.269091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714db64e-d2c5-4992-8814-08311838eb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:28.269736 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714db64e-d2c5-4992-8814-08311838eb0b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:28.270025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714db64e-d2c5-4992-8814-08311838eb0b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:28.270300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714db64e-d2c5-4992-8814-08311838eb0b None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:28.271543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714db64e-d2c5-4992-8814-08311838eb0b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:28.271867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714db64e-d2c5-4992-8814-08311838eb0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:28.273354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714db64e-d2c5-4992-8814-08311838eb0b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:28.283698 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 483/953] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Oct 6 12:09:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:28.286124 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76308a09-de24-430e-92fa-e2d58c77ffb5 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:28.407164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76308a09-de24-430e-92fa-e2d58c77ffb5 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:28.426345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:28.449385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b593b6d-ec26-43a9-810f-fd30a34f780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:28.450019 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b593b6d-ec26-43a9-810f-fd30a34f780b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:28.450295 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b593b6d-ec26-43a9-810f-fd30a34f780b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:28.450566 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b593b6d-ec26-43a9-810f-fd30a34f780b None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:28.451216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b593b6d-ec26-43a9-810f-fd30a34f780b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:28.451535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b593b6d-ec26-43a9-810f-fd30a34f780b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:28.453004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b593b6d-ec26-43a9-810f-fd30a34f780b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:28.458767 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 471/954] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Oct 6 12:09:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:28.556679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892cb9f-6c8c-49cc-8a0a-28ab975ace42 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:28.557048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892cb9f-6c8c-49cc-8a0a-28ab975ace42 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:28.557429 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892cb9f-6c8c-49cc-8a0a-28ab975ace42 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:28.559321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 484/955] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:28.565337 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0124881-b19d-4eba-8c18-05b311f91964 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:28.565831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0124881-b19d-4eba-8c18-05b311f91964 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:28.566189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0124881-b19d-4eba-8c18-05b311f91964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:28.586267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c0124881-b19d-4eba-8c18-05b311f91964 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:28.587552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c0124881-b19d-4eba-8c18-05b311f91964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:28.666671 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 472/956] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:28.933632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892cb9f-6c8c-49cc-8a0a-28ab975ace42 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:29.019951 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:29.041964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892cb9f-6c8c-49cc-8a0a-28ab975ace42 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:29.077367 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251a5ec-5c81-4783-a10b-fb7461542546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:29.078023 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251a5ec-5c81-4783-a10b-fb7461542546 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:29.078305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251a5ec-5c81-4783-a10b-fb7461542546 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:29.078580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251a5ec-5c81-4783-a10b-fb7461542546 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:29.079161 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251a5ec-5c81-4783-a10b-fb7461542546 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:29.080557 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251a5ec-5c81-4783-a10b-fb7461542546 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-f251a5ec-5c81-4783-a10b-fb7461542546 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:29.089528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:29.090670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:29.090670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:29.090670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:29.090670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:29.090670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:29.090670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:29.090670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:29.090670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:29.090670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:29.090670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:29.090670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:29.090670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:29.090670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:29.090670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:29.090670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:29.090670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:29.091639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:29.091639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:29.091639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:29.091639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:29.091639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:29.091639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:29.091639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:29.091639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:29.091639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:29.091639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:29.091639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:29.091639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:29.091639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 12:09:29.091639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:29.091639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 485/957] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Oct 6 12:09:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:29.096750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0124881-b19d-4eba-8c18-05b311f91964 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:29.201786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0124881-b19d-4eba-8c18-05b311f91964 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:29.212972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:29.237859 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e1665-70e6-44e5-81c6-99aa2e9b8fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:29.243693 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e1665-70e6-44e5-81c6-99aa2e9b8fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:29.243968 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e1665-70e6-44e5-81c6-99aa2e9b8fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:29.244238 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e1665-70e6-44e5-81c6-99aa2e9b8fc2 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:29.244882 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e1665-70e6-44e5-81c6-99aa2e9b8fc2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:29.245184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e1665-70e6-44e5-81c6-99aa2e9b8fc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:29.246631 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e1665-70e6-44e5-81c6-99aa2e9b8fc2 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:29.258156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 473/958] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Oct 6 12:09:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:29.263473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251a5ec-5c81-4783-a10b-fb7461542546 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:29.365851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251a5ec-5c81-4783-a10b-fb7461542546 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:29.400235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d1fb8-9797-40ca-bdf9-17ed069624aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:29.400849 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d1fb8-9797-40ca-bdf9-17ed069624aa None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:29.401129 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d1fb8-9797-40ca-bdf9-17ed069624aa None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:29.401398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d1fb8-9797-40ca-bdf9-17ed069624aa None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:29.402032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d1fb8-9797-40ca-bdf9-17ed069624aa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:29.402340 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d1fb8-9797-40ca-bdf9-17ed069624aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:29.403746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d1fb8-9797-40ca-bdf9-17ed069624aa None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:29.410083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 486/959] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Oct 6 12:09:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:29.416473 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e1665-70e6-44e5-81c6-99aa2e9b8fc2 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:29.521767 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e1665-70e6-44e5-81c6-99aa2e9b8fc2 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:29.556561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e9b87-aa8f-4a7a-8031-3b5e1ebc924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:29.557166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e9b87-aa8f-4a7a-8031-3b5e1ebc924f None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:29.557443 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e9b87-aa8f-4a7a-8031-3b5e1ebc924f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:29.557712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e9b87-aa8f-4a7a-8031-3b5e1ebc924f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:29.558379 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e9b87-aa8f-4a7a-8031-3b5e1ebc924f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:29.559809 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e9b87-aa8f-4a7a-8031-3b5e1ebc924f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:29.580796 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 474/960] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:09:29 2022] POST /identity/v3/endpoints => generated 349 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:09:29.847551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a821e3-b61c-4ef1-9a94-13c27b62d555 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:29.847903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a821e3-b61c-4ef1-9a94-13c27b62d555 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:29.848080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a821e3-b61c-4ef1-9a94-13c27b62d555 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:29.850153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 487/961] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:29.855591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06058be9-3cd4-439d-84be-0febcb73c0c2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:29.856150 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06058be9-3cd4-439d-84be-0febcb73c0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:29.856150 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06058be9-3cd4-439d-84be-0febcb73c0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:29.874345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-06058be9-3cd4-439d-84be-0febcb73c0c2 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:29.875554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-06058be9-3cd4-439d-84be-0febcb73c0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:29.901673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:29.957789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 475/962] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:30.229647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a821e3-b61c-4ef1-9a94-13c27b62d555 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:30.244504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:30.347488 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a821e3-b61c-4ef1-9a94-13c27b62d555 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:30.382802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb9db0-8058-4b84-a411-e9d42869b6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:30.383472 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb9db0-8058-4b84-a411-e9d42869b6b0 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:30.383775 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb9db0-8058-4b84-a411-e9d42869b6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:30.384049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb9db0-8058-4b84-a411-e9d42869b6b0 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:30.384700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb9db0-8058-4b84-a411-e9d42869b6b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:30.386132 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb9db0-8058-4b84-a411-e9d42869b6b0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:30.402738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.sql.core [None req-cacb9db0-8058-4b84-a411-e9d42869b6b0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 06 12:09:30.402738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 06 12:09:30.402738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [parameters: {'user_id': 'e7a21e83968d4cec92ad25ae677ceb40', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 06 12:09:30.402738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81355) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 06 12:09:30.410546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-cacb9db0-8058-4b84-a411-e9d42869b6b0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 06 12:09:30.410546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:30.410546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 12:09:30.410546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 12:09:30.410546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 12:09:30.410546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 12:09:30.410546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 12:09:30.410546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 12:09:30.410546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 12:09:30.410546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application conn.query(q) Oct 06 12:09:30.410546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 12:09:30.410546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 12:09:30.410546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 12:09:30.410546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application result.read() Oct 06 12:09:30.410546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 12:09:30.412496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application next(self.gen) Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application yield resource Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application next(self.gen) Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self.session.rollback() Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self.force_reraise() Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise self.value Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application session.commit() Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 06 12:09:30.413786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._prepare_impl() Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self.session.flush() Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._flush(objects) Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application compat.raise_( Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application flush_context.execute() Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rec.execute(self) Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 06 12:09:30.414964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 06 12:09:30.416027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 06 12:09:30.416027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 06 12:09:30.416027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 06 12:09:30.416027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 06 12:09:30.416027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 06 12:09:30.416027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 06 12:09:30.416027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 06 12:09:30.416027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 06 12:09:30.416027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 06 12:09:30.416027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 06 12:09:30.416027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 06 12:09:30.416027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 06 12:09:30.416027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 06 12:09:30.416027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 06 12:09:30.416027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 12:09:30.416027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception Oct 06 12:09:30.416885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 12:09:30.416885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 12:09:30.416885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 12:09:30.416885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 12:09:30.416885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 12:09:30.416885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 12:09:30.416885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 12:09:30.416885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application conn.query(q) Oct 06 12:09:30.416885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 12:09:30.416885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 12:09:30.416885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 12:09:30.416885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application result.read() Oct 06 12:09:30.416885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 12:09:30.416885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 12:09:30.416885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 12:09:30.416885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 12:09:30.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 12:09:30.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 12:09:30.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 12:09:30.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 12:09:30.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 06 12:09:30.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 06 12:09:30.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e7a21e83968d4cec92ad25ae677ceb40', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 06 12:09:30.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 06 12:09:30.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:30.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:30.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:30.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:30.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:30.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:30.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 06 12:09:30.418538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 06 12:09:30.426529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 06 12:09:30.426529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 06 12:09:30.426529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 06 12:09:30.426529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 06 12:09:30.426529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 06 12:09:30.426529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:30.426529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 488/963] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Oct 6 12:09:30 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:30.428157 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06058be9-3cd4-439d-84be-0febcb73c0c2 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:30.428891 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fe3b4-24d6-43fa-8ddf-cb7759dc4455 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:30.429183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fe3b4-24d6-43fa-8ddf-cb7759dc4455 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:30.429454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fe3b4-24d6-43fa-8ddf-cb7759dc4455 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:30.434291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 489/964] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:30.439409 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b946d96-7769-4311-bff2-a2ef7ee245b3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:30.439712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b946d96-7769-4311-bff2-a2ef7ee245b3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:30.439987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b946d96-7769-4311-bff2-a2ef7ee245b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:30.459627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8b946d96-7769-4311-bff2-a2ef7ee245b3 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:30.461302 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8b946d96-7769-4311-bff2-a2ef7ee245b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:30.534938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:30.542450 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 490/965] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:30.555098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06058be9-3cd4-439d-84be-0febcb73c0c2 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:30.591378 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d265c-f44d-40b7-bb72-7325193803b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:30.592376 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d265c-f44d-40b7-bb72-7325193803b7 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:30.592850 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d265c-f44d-40b7-bb72-7325193803b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:30.593313 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d265c-f44d-40b7-bb72-7325193803b7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:30.601469 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d265c-f44d-40b7-bb72-7325193803b7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:30.603218 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d265c-f44d-40b7-bb72-7325193803b7 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:30.615227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-6c3d265c-f44d-40b7-bb72-7325193803b7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 12:09:30.615227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:30.615227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:30.615227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:30.615227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:30.615227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:30.615227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:30.615227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:30.615227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:30.615227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:30.615227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:30.615227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:30.615227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:09:30.615227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:09:30.615227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:09:30.615227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:30.616191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:30.617174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:30.617174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:30.617174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:09:30.617174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:09:30.617174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:09:30.617174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:09:30.617174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:09:30.617174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:09:30.617174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 12:09:30.617174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:30.619825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 476/966] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:09:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:30.627537 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b946d96-7769-4311-bff2-a2ef7ee245b3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:30.732133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b946d96-7769-4311-bff2-a2ef7ee245b3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:30.767210 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a11a7-72c7-4641-9223-04e4bc93a7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:30.767846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a11a7-72c7-4641-9223-04e4bc93a7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:30.768127 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a11a7-72c7-4641-9223-04e4bc93a7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:30.768398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a11a7-72c7-4641-9223-04e4bc93a7a5 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:30.769098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a11a7-72c7-4641-9223-04e4bc93a7a5 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:30.769400 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a11a7-72c7-4641-9223-04e4bc93a7a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:30.771101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a11a7-72c7-4641-9223-04e4bc93a7a5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:30.790899 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b4759-710b-4df6-8d29-a901d1792f3c None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:30.791184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b4759-710b-4df6-8d29-a901d1792f3c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:30.791358 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b4759-710b-4df6-8d29-a901d1792f3c None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:30.793978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 477/967] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:09:30.797783 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbc65e-484c-4e26-971b-bd7692dfca49 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:30.798064 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbc65e-484c-4e26-971b-bd7692dfca49 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:30.798336 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbc65e-484c-4e26-971b-bd7692dfca49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:30.809746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 491/968] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Oct 6 12:09:30 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:30.813203 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a11a7-72c7-4641-9223-04e4bc93a7a5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:30.823873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-4dbbc65e-484c-4e26-971b-bd7692dfca49 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:30.825079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4dbbc65e-484c-4e26-971b-bd7692dfca49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:30.848494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:30.886967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:30.918617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 478/969] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:30.941739 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a11a7-72c7-4641-9223-04e4bc93a7a5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:30.982358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e42059-ba6d-4e0f-a6ff-59c8714e3bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:30.983373 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e42059-ba6d-4e0f-a6ff-59c8714e3bbb None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:30.983884 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e42059-ba6d-4e0f-a6ff-59c8714e3bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:30.984385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e42059-ba6d-4e0f-a6ff-59c8714e3bbb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:30.997362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e42059-ba6d-4e0f-a6ff-59c8714e3bbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:31.002636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-84e42059-ba6d-4e0f-a6ff-59c8714e3bbb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 12:09:31.002636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:31.002636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:31.002636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:31.002636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:31.002636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:31.002636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:31.002636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:31.002636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:31.002636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:31.002636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:31.002636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:31.002636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:31.002636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:31.002636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:31.002636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:31.003646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:31.003646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:31.003646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:31.003646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:31.003646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:31.003646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:31.003646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:31.003646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:31.003646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:31.003646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:31.003646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:31.003646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:31.003646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:31.003646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:31.003646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:31.003646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:31.004647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:31.004647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:31.004647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:31.004647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:31.004647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:31.004647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:31.004647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:31.004647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:31.004647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:31.004647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:31.004647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 12:09:31.004647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:31.006827 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 492/970] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Oct 6 12:09:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:31.013461 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbc65e-484c-4e26-971b-bd7692dfca49 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:31.121573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbc65e-484c-4e26-971b-bd7692dfca49 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:31.157689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffa021-26fe-4d8a-aef1-e85e72aff5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:31.158304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffa021-26fe-4d8a-aef1-e85e72aff5db None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:31.158586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffa021-26fe-4d8a-aef1-e85e72aff5db None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:31.158856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffa021-26fe-4d8a-aef1-e85e72aff5db None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:31.159534 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffa021-26fe-4d8a-aef1-e85e72aff5db None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:31.159834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffa021-26fe-4d8a-aef1-e85e72aff5db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:31.161193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffa021-26fe-4d8a-aef1-e85e72aff5db None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:31.169553 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 479/971] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Oct 6 12:09:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:31.175199 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e42059-ba6d-4e0f-a6ff-59c8714e3bbb None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:31.193358 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffa021-26fe-4d8a-aef1-e85e72aff5db None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:31.204621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:31.292251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e42059-ba6d-4e0f-a6ff-59c8714e3bbb None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:31.297634 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffa021-26fe-4d8a-aef1-e85e72aff5db None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:31.312548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:31.330892 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c95fa-1f7f-4d15-aef1-a271c4d3331e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:31.331548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c95fa-1f7f-4d15-aef1-a271c4d3331e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:31.331830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c95fa-1f7f-4d15-aef1-a271c4d3331e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:31.332100 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c95fa-1f7f-4d15-aef1-a271c4d3331e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:31.333464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb92cf7-1ae1-42dd-b4cb-70f1c5e56a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:31.334228 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb92cf7-1ae1-42dd-b4cb-70f1c5e56a5a None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:31.334511 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb92cf7-1ae1-42dd-b4cb-70f1c5e56a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:31.335093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb92cf7-1ae1-42dd-b4cb-70f1c5e56a5a None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:31.335785 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb92cf7-1ae1-42dd-b4cb-70f1c5e56a5a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:31.337148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb92cf7-1ae1-42dd-b4cb-70f1c5e56a5a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:31.342252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c95fa-1f7f-4d15-aef1-a271c4d3331e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:31.343769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c95fa-1f7f-4d15-aef1-a271c4d3331e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:31.354798 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 480/972] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Oct 6 12:09:31 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:09:31.359860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-651c95fa-1f7f-4d15-aef1-a271c4d3331e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:31.359860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:31.359860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:31.359860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:31.359860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:31.359860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:31.359860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:31.359860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:31.359860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:31.359860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:31.359860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:31.359860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:31.359860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:31.359860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:31.359860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:31.359860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:31.360954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:31.361878 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:31.361878 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:31.361878 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:31.365767 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 493/973] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:31.366931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb92cf7-1ae1-42dd-b4cb-70f1c5e56a5a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:31.438500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:31.484765 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb92cf7-1ae1-42dd-b4cb-70f1c5e56a5a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:31.522307 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d13bb-2a4d-4361-9578-5bfb7650ee08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:31.522924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d13bb-2a4d-4361-9578-5bfb7650ee08 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:31.523201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d13bb-2a4d-4361-9578-5bfb7650ee08 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:31.523474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d13bb-2a4d-4361-9578-5bfb7650ee08 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:31.525496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d13bb-2a4d-4361-9578-5bfb7650ee08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:31.525496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d13bb-2a4d-4361-9578-5bfb7650ee08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:31.525824 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d13bb-2a4d-4361-9578-5bfb7650ee08 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:31.538463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 481/974] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:31.544335 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c95fa-1f7f-4d15-aef1-a271c4d3331e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:31.647142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c95fa-1f7f-4d15-aef1-a271c4d3331e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:31.680402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614ac18-3160-4da0-9764-993542ded87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:31.681030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614ac18-3160-4da0-9764-993542ded87b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:31.681311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614ac18-3160-4da0-9764-993542ded87b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:31.681581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614ac18-3160-4da0-9764-993542ded87b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:31.689219 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1614ac18-3160-4da0-9764-993542ded87b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:31.690600 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1614ac18-3160-4da0-9764-993542ded87b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:31.701589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-1614ac18-3160-4da0-9764-993542ded87b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 12:09:31.701589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:31.701589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:31.701589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:31.701589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:31.701589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:31.701589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:31.701589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:31.701589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:31.701589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:31.701589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:31.701589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:31.701589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:09:31.701589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:09:31.701589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:09:31.701589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:31.702644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:31.708917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:31.708917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:31.708917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:09:31.708917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:09:31.708917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:09:31.708917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:09:31.708917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:09:31.708917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:09:31.708917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 12:09:31.708917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:31.708917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 494/975] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Oct 6 12:09:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:31.711134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d13bb-2a4d-4361-9578-5bfb7650ee08 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:31.818934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d13bb-2a4d-4361-9578-5bfb7650ee08 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:31.842129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:31.854503 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c69cbd-c00f-46e3-b6aa-c59127e5a13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:31.855112 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c69cbd-c00f-46e3-b6aa-c59127e5a13f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:31.855389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c69cbd-c00f-46e3-b6aa-c59127e5a13f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:31.855689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c69cbd-c00f-46e3-b6aa-c59127e5a13f None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:31.856417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c69cbd-c00f-46e3-b6aa-c59127e5a13f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:31.856716 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c69cbd-c00f-46e3-b6aa-c59127e5a13f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:31.858131 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c69cbd-c00f-46e3-b6aa-c59127e5a13f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:31.862241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c4ce3-234b-4227-b2d1-8393937b664a None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:31.862526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c4ce3-234b-4227-b2d1-8393937b664a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:31.862799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c4ce3-234b-4227-b2d1-8393937b664a None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:31.864319 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 495/976] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:31.869092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58ebf0-582f-4ad4-8870-79871965a968 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:31.869374 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58ebf0-582f-4ad4-8870-79871965a968 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:31.869653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58ebf0-582f-4ad4-8870-79871965a968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:31.879192 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 482/977] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Oct 6 12:09:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:31.884711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c69cbd-c00f-46e3-b6aa-c59127e5a13f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:31.891205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-7e58ebf0-582f-4ad4-8870-79871965a968 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:31.892413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7e58ebf0-582f-4ad4-8870-79871965a968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:31.917697 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:31.964582 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 496/978] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:31.987517 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c69cbd-c00f-46e3-b6aa-c59127e5a13f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:32.021872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c034b-8de9-4f27-929f-1388d48a0133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:32.022602 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c034b-8de9-4f27-929f-1388d48a0133 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:32.022982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c034b-8de9-4f27-929f-1388d48a0133 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:32.023357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c034b-8de9-4f27-929f-1388d48a0133 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:32.071168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:32.090014 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c034b-8de9-4f27-929f-1388d48a0133 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:32.091466 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c034b-8de9-4f27-929f-1388d48a0133 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:32.188589 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 483/979] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:09:31 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 06 12:09:32.194612 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58ebf0-582f-4ad4-8870-79871965a968 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:32.230124 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c034b-8de9-4f27-929f-1388d48a0133 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:32.315225 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58ebf0-582f-4ad4-8870-79871965a968 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:32.322943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:32.343696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c034b-8de9-4f27-929f-1388d48a0133 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:32.356179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c218be-9cf5-4629-ad4d-dd9ddc008f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:32.356810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c218be-9cf5-4629-ad4d-dd9ddc008f9d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:32.357095 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c218be-9cf5-4629-ad4d-dd9ddc008f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:32.357482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c218be-9cf5-4629-ad4d-dd9ddc008f9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:32.368922 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c218be-9cf5-4629-ad4d-dd9ddc008f9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:32.370361 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c218be-9cf5-4629-ad4d-dd9ddc008f9d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:32.379259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a3a3c-2db2-48b2-90ae-e04085563b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:32.379925 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a3a3c-2db2-48b2-90ae-e04085563b19 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:32.380208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a3a3c-2db2-48b2-90ae-e04085563b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:32.380482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a3a3c-2db2-48b2-90ae-e04085563b19 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:32.381042 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a3a3c-2db2-48b2-90ae-e04085563b19 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:32.382380 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a3a3c-2db2-48b2-90ae-e04085563b19 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:32.385576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-94c218be-9cf5-4629-ad4d-dd9ddc008f9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:32.385576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:32.385576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:32.385576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:32.385576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:32.385576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:32.385576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:32.385576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:32.385576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:32.385576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:32.385576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:32.385576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:32.385576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:32.385576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:32.385576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:32.385576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:32.386517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:32.387431 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:32.387431 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:32.387431 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:32.387431 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 497/980] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:32.392121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c218be-9cf5-4629-ad4d-dd9ddc008f9d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-537a3a3c-2db2-48b2-90ae-e04085563b19 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:32.393598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:32.394604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:32.394604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:32.394604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:32.394604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:32.394604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:32.394604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:32.394604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:32.394604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:32.394604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:32.394604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:32.394604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:32.394604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:32.394604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:32.394604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:32.394604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:32.394604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:32.395464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:32.395464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:32.395464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:32.395464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:32.395464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:32.395464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:32.395464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:32.395464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:32.395464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:32.395464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:32.395464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:32.395464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:32.395464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 12:09:32.395464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:32.399557 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 484/981] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Oct 6 12:09:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:32.400560 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a3a3c-2db2-48b2-90ae-e04085563b19 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:32.409962 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:32.503201 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c218be-9cf5-4629-ad4d-dd9ddc008f9d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:32.503390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a3a3c-2db2-48b2-90ae-e04085563b19 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:32.537293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215f7e6-d499-40f6-a5e5-004e4bb807f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:32.537896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215f7e6-d499-40f6-a5e5-004e4bb807f6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:32.538172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215f7e6-d499-40f6-a5e5-004e4bb807f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:32.538444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215f7e6-d499-40f6-a5e5-004e4bb807f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:32.539065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215f7e6-d499-40f6-a5e5-004e4bb807f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:32.539362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215f7e6-d499-40f6-a5e5-004e4bb807f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:32.539946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7c684-d163-4ce2-af5a-0bcf9e614b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:32.540553 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7c684-d163-4ce2-af5a-0bcf9e614b62 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:32.540736 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215f7e6-d499-40f6-a5e5-004e4bb807f6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:32.540841 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7c684-d163-4ce2-af5a-0bcf9e614b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:32.541091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7c684-d163-4ce2-af5a-0bcf9e614b62 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:32.541712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf7c684-d163-4ce2-af5a-0bcf9e614b62 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:32.542017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf7c684-d163-4ce2-af5a-0bcf9e614b62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:32.543527 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf7c684-d163-4ce2-af5a-0bcf9e614b62 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:32.548411 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 485/982] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Oct 6 12:09:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:32.555543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 498/983] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:32.556769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7c684-d163-4ce2-af5a-0bcf9e614b62 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:32.559153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215f7e6-d499-40f6-a5e5-004e4bb807f6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:32.570962 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:32.634994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:32.661572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7c684-d163-4ce2-af5a-0bcf9e614b62 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:32.666413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215f7e6-d499-40f6-a5e5-004e4bb807f6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:32.697472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e0bef-b7e3-4062-a898-dbceb5ae5f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:32.698081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0bef-b7e3-4062-a898-dbceb5ae5f81 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:32.698365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0bef-b7e3-4062-a898-dbceb5ae5f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:32.698634 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0bef-b7e3-4062-a898-dbceb5ae5f81 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:32.699257 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e0bef-b7e3-4062-a898-dbceb5ae5f81 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:32.699648 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e0bef-b7e3-4062-a898-dbceb5ae5f81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:32.701013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e0bef-b7e3-4062-a898-dbceb5ae5f81 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:32.703498 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0763d-4ecf-401a-a4f0-ff7625a8babb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:32.704136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0763d-4ecf-401a-a4f0-ff7625a8babb None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:32.704418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0763d-4ecf-401a-a4f0-ff7625a8babb None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:32.704680 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0763d-4ecf-401a-a4f0-ff7625a8babb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:32.706763 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 486/984] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Oct 6 12:09:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:32.712458 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e0bef-b7e3-4062-a898-dbceb5ae5f81 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:32.712609 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0763d-4ecf-401a-a4f0-ff7625a8babb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:32.714236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0763d-4ecf-401a-a4f0-ff7625a8babb None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:32.728323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-a9d0763d-4ecf-401a-a4f0-ff7625a8babb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:32.728323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:32.728323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:32.728323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:32.728323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:32.728323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:32.728323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:32.728323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:32.728323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:32.728323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:32.728323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:32.728323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:32.728323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:09:32.728323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:09:32.728323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:32.729226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:32.730252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:09:32.730252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:09:32.730252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:09:32.730252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:09:32.730252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:09:32.730252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:09:32.730252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:32.730252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:32.734540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 499/985] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:32.735310 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0763d-4ecf-401a-a4f0-ff7625a8babb None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:32.790998 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:32.815743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e0bef-b7e3-4062-a898-dbceb5ae5f81 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:32.846986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0763d-4ecf-401a-a4f0-ff7625a8babb None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:32.850799 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ffc91-dbda-40d4-b4f7-79b8eedd097f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:32.851409 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ffc91-dbda-40d4-b4f7-79b8eedd097f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:32.851722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ffc91-dbda-40d4-b4f7-79b8eedd097f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:32.852071 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ffc91-dbda-40d4-b4f7-79b8eedd097f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:32.852778 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ffc91-dbda-40d4-b4f7-79b8eedd097f None admin] RBAC: Adding query filter params (interface=admin, service_id=8df23e93489d4b119af080728e844df6, region_id=RegionOne) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:32.853088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ffc91-dbda-40d4-b4f7-79b8eedd097f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:32.854539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ffc91-dbda-40d4-b4f7-79b8eedd097f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:32.860043 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 487/986] 10.222.0.33 () {56 vars in 1249 bytes} [Thu Oct 6 12:09:32 2022] GET /identity/v3/endpoints?service_id=8df23e93489d4b119af080728e844df6&interface=admin®ion_id=RegionOne => generated 193 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:32.866278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ffc91-dbda-40d4-b4f7-79b8eedd097f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:32.883299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:32.890709 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a3f77-ba7e-4774-b10b-200568c48b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:32.891320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a3f77-ba7e-4774-b10b-200568c48b57 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:32.891627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a3f77-ba7e-4774-b10b-200568c48b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:32.892284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a3f77-ba7e-4774-b10b-200568c48b57 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:32.893030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a3f77-ba7e-4774-b10b-200568c48b57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:32.893292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a3f77-ba7e-4774-b10b-200568c48b57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:32.894757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a3f77-ba7e-4774-b10b-200568c48b57 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:32.912345 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 500/987] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:09:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:32.914302 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a3f77-ba7e-4774-b10b-200568c48b57 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:32.978174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ffc91-dbda-40d4-b4f7-79b8eedd097f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:33.014115 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a7255-42c6-41ab-8eab-542bc48d98e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:33.014808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a7255-42c6-41ab-8eab-542bc48d98e9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:33.015094 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a7255-42c6-41ab-8eab-542bc48d98e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:33.015366 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a7255-42c6-41ab-8eab-542bc48d98e9 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:33.015997 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a7255-42c6-41ab-8eab-542bc48d98e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:33.017363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a7255-42c6-41ab-8eab-542bc48d98e9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:33.022278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a3f77-ba7e-4774-b10b-200568c48b57 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:33.023601 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 488/988] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Oct 6 12:09:32 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:09:33.069840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4abe29-81f0-475a-b41f-c2d9a26d3768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:33.070367 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4abe29-81f0-475a-b41f-c2d9a26d3768 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:33.070645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4abe29-81f0-475a-b41f-c2d9a26d3768 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:33.070910 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4abe29-81f0-475a-b41f-c2d9a26d3768 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:33.071666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4abe29-81f0-475a-b41f-c2d9a26d3768 None admin] RBAC: Adding query filter params (role.id=14d78f802e8d43d684535749f98404c7, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=96a302378dc0416d87c1cf11a3ba0a9c) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:33.071973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4abe29-81f0-475a-b41f-c2d9a26d3768 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:33.073431 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4abe29-81f0-475a-b41f-c2d9a26d3768 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:33.087816 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 501/989] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:09:32 2022] GET /identity/v3/role_assignments?user.id=96a302378dc0416d87c1cf11a3ba0a9c&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=14d78f802e8d43d684535749f98404c7 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:33.587070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5d9b3-d42b-4ff7-98f2-813ee7aedbe7 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:33.587468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5d9b3-d42b-4ff7-98f2-813ee7aedbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:33.587694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5d9b3-d42b-4ff7-98f2-813ee7aedbe7 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:33.588978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 489/990] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:33.594275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0b582-328d-480a-bcf0-e8236d26559c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:33.594718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0b582-328d-480a-bcf0-e8236d26559c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:33.594795 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0b582-328d-480a-bcf0-e8236d26559c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:33.613595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fca0b582-328d-480a-bcf0-e8236d26559c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:33.614855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fca0b582-328d-480a-bcf0-e8236d26559c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:33.666929 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:33.685348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 502/991] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:33.955309 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5d9b3-d42b-4ff7-98f2-813ee7aedbe7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:33.959004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:34.062033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5d9b3-d42b-4ff7-98f2-813ee7aedbe7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:34.095043 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280e1af-669d-46c9-947a-d9a11d7566da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:34.095687 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280e1af-669d-46c9-947a-d9a11d7566da None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:34.095967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280e1af-669d-46c9-947a-d9a11d7566da None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:34.096236 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280e1af-669d-46c9-947a-d9a11d7566da None admin] PATH_INFO: `/v3/services/compute` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:34.096808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280e1af-669d-46c9-947a-d9a11d7566da None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:34.098150 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280e1af-669d-46c9-947a-d9a11d7566da None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-b280e1af-669d-46c9-947a-d9a11d7566da None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:34.107137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:34.108164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:34.108164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:34.108164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:34.108164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:34.108164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:34.108164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:34.108164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:34.108164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:34.108164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:34.108164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:34.108164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:34.108164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:34.108164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:34.108164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:34.108164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:34.108164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:34.109021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:34.109021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:34.109021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:34.109021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:34.109021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:34.109021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:34.109021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:34.109021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:34.109021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:34.109021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:34.109021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:34.109021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:34.109021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 12:09:34.109021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:34.111813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 490/992] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:34.114043 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0b582-328d-480a-bcf0-e8236d26559c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:34.213310 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0b582-328d-480a-bcf0-e8236d26559c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:34.256578 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c32363-b6a4-42c6-9cec-459ab4a9cd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:34.257190 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c32363-b6a4-42c6-9cec-459ab4a9cd22 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:34.257468 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c32363-b6a4-42c6-9cec-459ab4a9cd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:34.257734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c32363-b6a4-42c6-9cec-459ab4a9cd22 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:34.258364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c32363-b6a4-42c6-9cec-459ab4a9cd22 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:34.258654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c32363-b6a4-42c6-9cec-459ab4a9cd22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:34.260036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c32363-b6a4-42c6-9cec-459ab4a9cd22 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:34.269362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 503/993] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Oct 6 12:09:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:34.271258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280e1af-669d-46c9-947a-d9a11d7566da None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:34.340810 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:34.376571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280e1af-669d-46c9-947a-d9a11d7566da None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:34.410801 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91fb32-0d8e-43da-a9b3-27f47881eef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:34.411947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91fb32-0d8e-43da-a9b3-27f47881eef9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:34.412230 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91fb32-0d8e-43da-a9b3-27f47881eef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:34.412501 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91fb32-0d8e-43da-a9b3-27f47881eef9 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:34.413120 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91fb32-0d8e-43da-a9b3-27f47881eef9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:34.413421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91fb32-0d8e-43da-a9b3-27f47881eef9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:34.414776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91fb32-0d8e-43da-a9b3-27f47881eef9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:34.419859 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 491/994] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Oct 6 12:09:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:34.425416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c32363-b6a4-42c6-9cec-459ab4a9cd22 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:34.475142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:34.524823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c32363-b6a4-42c6-9cec-459ab4a9cd22 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:34.557512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b4450-bc8f-495b-98c9-d6c58026e233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:34.558116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b4450-bc8f-495b-98c9-d6c58026e233 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:34.558395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b4450-bc8f-495b-98c9-d6c58026e233 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:34.558662 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b4450-bc8f-495b-98c9-d6c58026e233 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:34.559402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b4450-bc8f-495b-98c9-d6c58026e233 None admin] RBAC: Adding query filter params (interface=public, service_id=4e6aebb446224ffeafd2b5c1d8a95023, region_id=RegionOne) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:34.559738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b4450-bc8f-495b-98c9-d6c58026e233 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:34.561089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b4450-bc8f-495b-98c9-d6c58026e233 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:34.566066 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 504/995] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Oct 6 12:09:34 2022] GET /identity/v3/endpoints?service_id=4e6aebb446224ffeafd2b5c1d8a95023&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:34.572035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91fb32-0d8e-43da-a9b3-27f47881eef9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:34.673056 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91fb32-0d8e-43da-a9b3-27f47881eef9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:34.702415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:34.709332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d12e6e-b47f-45ca-bd56-0b572c0fd419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:34.709945 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d12e6e-b47f-45ca-bd56-0b572c0fd419 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:34.710224 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d12e6e-b47f-45ca-bd56-0b572c0fd419 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:34.710493 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d12e6e-b47f-45ca-bd56-0b572c0fd419 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:34.711087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d12e6e-b47f-45ca-bd56-0b572c0fd419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:34.712524 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d12e6e-b47f-45ca-bd56-0b572c0fd419 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:34.727864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 492/996] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Oct 6 12:09:34 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:09:35.194716 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956138d-a9c3-4106-821a-6812fbc7cbdd None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:35.195272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956138d-a9c3-4106-821a-6812fbc7cbdd None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:35.195272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956138d-a9c3-4106-821a-6812fbc7cbdd None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:35.196799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 505/997] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:35.202310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42872e4a-ed4b-4d37-949b-a4bd46aad650 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:35.202751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42872e4a-ed4b-4d37-949b-a4bd46aad650 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:35.202863 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42872e4a-ed4b-4d37-949b-a4bd46aad650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:35.222088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-42872e4a-ed4b-4d37-949b-a4bd46aad650 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:35.223270 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-42872e4a-ed4b-4d37-949b-a4bd46aad650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:35.286331 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378a66c-5620-4016-803c-579c20d9e17c None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:35.286611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378a66c-5620-4016-803c-579c20d9e17c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:35.286887 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378a66c-5620-4016-803c-579c20d9e17c None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:35.288383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 506/998] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:35.293254 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977f771-68d4-4338-a828-2da17239f47d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:35.293535 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977f771-68d4-4338-a828-2da17239f47d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:35.293806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977f771-68d4-4338-a828-2da17239f47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:35.297810 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 493/999] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:35.313927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b977f771-68d4-4338-a828-2da17239f47d None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:35.315400 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b977f771-68d4-4338-a828-2da17239f47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:35.382239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 507/1000] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:35.563328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42872e4a-ed4b-4d37-949b-a4bd46aad650 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:35.646160 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977f771-68d4-4338-a828-2da17239f47d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:35.669091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42872e4a-ed4b-4d37-949b-a4bd46aad650 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:35.694457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:35.696080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:35.705953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340de86-a462-4980-9daa-2c187ff88e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:35.706587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340de86-a462-4980-9daa-2c187ff88e9d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:35.706869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340de86-a462-4980-9daa-2c187ff88e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:35.707139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340de86-a462-4980-9daa-2c187ff88e9d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:35.707744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340de86-a462-4980-9daa-2c187ff88e9d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:35.709152 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340de86-a462-4980-9daa-2c187ff88e9d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-8340de86-a462-4980-9daa-2c187ff88e9d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:35.718382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:35.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:35.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:35.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:35.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:35.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:35.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:35.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:35.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:35.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:35.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:35.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:35.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:35.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:35.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:35.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:35.719388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:35.721017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:35.721017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:35.721017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:35.721017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:35.721017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:35.721017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:35.721017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:35.721017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:35.721017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:35.721017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:35.721017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:35.721017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:35.721017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 12:09:35.721017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:35.721017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 494/1001] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Oct 6 12:09:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:35.726060 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340de86-a462-4980-9daa-2c187ff88e9d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:35.755048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977f771-68d4-4338-a828-2da17239f47d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:35.800274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fd640-970e-4985-8971-1292b85fc441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:35.800908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fd640-970e-4985-8971-1292b85fc441 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:35.801191 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fd640-970e-4985-8971-1292b85fc441 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:35.801464 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fd640-970e-4985-8971-1292b85fc441 None admin] PATH_INFO: `/v3/services/image` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:35.802561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742fd640-970e-4985-8971-1292b85fc441 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:35.803957 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742fd640-970e-4985-8971-1292b85fc441 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-742fd640-970e-4985-8971-1292b85fc441 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:35.812541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:35.813639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:35.813639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:35.813639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:35.813639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:35.813639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:35.813639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:35.813639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:35.813639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:35.813639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:35.813639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:35.813639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:35.813639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:35.813639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:35.813639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:35.813639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:35.813639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:35.814547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:35.814547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:35.814547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:35.814547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:35.814547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:35.814547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:35.814547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:35.814547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:35.814547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:35.814547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:35.814547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:35.814547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:35.814547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 12:09:35.814547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:35.814547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 508/1002] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Oct 6 12:09:35 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:35.819368 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fd640-970e-4985-8971-1292b85fc441 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:35.844676 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340de86-a462-4980-9daa-2c187ff88e9d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:35.882033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c9b29-d2fc-4cc2-9498-9912f0247a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:35.882638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c9b29-d2fc-4cc2-9498-9912f0247a40 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:35.882924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c9b29-d2fc-4cc2-9498-9912f0247a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:35.883194 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c9b29-d2fc-4cc2-9498-9912f0247a40 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:35.883853 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c9b29-d2fc-4cc2-9498-9912f0247a40 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:35.884158 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c9b29-d2fc-4cc2-9498-9912f0247a40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:35.885516 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c9b29-d2fc-4cc2-9498-9912f0247a40 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:35.895591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 495/1003] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Oct 6 12:09:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:35.896939 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c9b29-d2fc-4cc2-9498-9912f0247a40 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:35.937335 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fd640-970e-4985-8971-1292b85fc441 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:35.947002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:35.956706 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:35.975301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ab8c3-c5b9-48b0-a34b-0a4150df131b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:35.976000 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ab8c3-c5b9-48b0-a34b-0a4150df131b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:35.976285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ab8c3-c5b9-48b0-a34b-0a4150df131b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:35.976562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ab8c3-c5b9-48b0-a34b-0a4150df131b None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:35.977194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4ab8c3-c5b9-48b0-a34b-0a4150df131b None admin] RBAC: Adding query filter params (name=image) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:35.977503 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4ab8c3-c5b9-48b0-a34b-0a4150df131b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:35.978871 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4ab8c3-c5b9-48b0-a34b-0a4150df131b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:35.988288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 509/1004] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:35.989833 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ab8c3-c5b9-48b0-a34b-0a4150df131b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:36.010988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c9b29-d2fc-4cc2-9498-9912f0247a40 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:36.045775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbf249-bc4e-4000-bb03-ff87c2bcfa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:36.046799 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbf249-bc4e-4000-bb03-ff87c2bcfa56 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:36.047089 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbf249-bc4e-4000-bb03-ff87c2bcfa56 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:36.047362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbf249-bc4e-4000-bb03-ff87c2bcfa56 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:36.048020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fbf249-bc4e-4000-bb03-ff87c2bcfa56 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:36.048414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fbf249-bc4e-4000-bb03-ff87c2bcfa56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:36.049786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fbf249-bc4e-4000-bb03-ff87c2bcfa56 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:36.055130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 496/1005] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Oct 6 12:09:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:36.060746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbf249-bc4e-4000-bb03-ff87c2bcfa56 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:36.101849 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ab8c3-c5b9-48b0-a34b-0a4150df131b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:36.140363 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312bc4f-d6fe-47fa-8bd1-ce35082e7343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:36.140970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312bc4f-d6fe-47fa-8bd1-ce35082e7343 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:36.141252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312bc4f-d6fe-47fa-8bd1-ce35082e7343 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:36.141524 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312bc4f-d6fe-47fa-8bd1-ce35082e7343 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:36.142145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312bc4f-d6fe-47fa-8bd1-ce35082e7343 None admin] RBAC: Adding query filter params (type=image) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:36.142446 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312bc4f-d6fe-47fa-8bd1-ce35082e7343 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:36.143836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312bc4f-d6fe-47fa-8bd1-ce35082e7343 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:36.148389 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 510/1006] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:36.170079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbf249-bc4e-4000-bb03-ff87c2bcfa56 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:36.188676 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:36.209037 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f208091-64c9-4a7d-b4ac-f1f38ab69d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:36.209735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f208091-64c9-4a7d-b4ac-f1f38ab69d31 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:36.209964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f208091-64c9-4a7d-b4ac-f1f38ab69d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:36.210209 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f208091-64c9-4a7d-b4ac-f1f38ab69d31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:36.210898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f208091-64c9-4a7d-b4ac-f1f38ab69d31 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:36.212343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f208091-64c9-4a7d-b4ac-f1f38ab69d31 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:36.234393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 497/1007] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:09:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:09:36.892715 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cc1af-9256-4ff9-b9c5-346732d51f8b None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:36.893281 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cc1af-9256-4ff9-b9c5-346732d51f8b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:36.893281 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cc1af-9256-4ff9-b9c5-346732d51f8b None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:36.894749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 511/1008] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:09:36.900207 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186453f3-d886-428b-9bcf-fe44a5ea8e74 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:36.900562 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186453f3-d886-428b-9bcf-fe44a5ea8e74 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:36.900898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186453f3-d886-428b-9bcf-fe44a5ea8e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:36.919559 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-186453f3-d886-428b-9bcf-fe44a5ea8e74 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:36.920739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-186453f3-d886-428b-9bcf-fe44a5ea8e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:36.995296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 498/1009] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:36 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:37.258945 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cc1af-9256-4ff9-b9c5-346732d51f8b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:37.296356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:37.361454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cc1af-9256-4ff9-b9c5-346732d51f8b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:37.394593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1693b5f-3e12-4e63-ba29-4823ad7da78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:37.395326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1693b5f-3e12-4e63-ba29-4823ad7da78c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:37.396159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1693b5f-3e12-4e63-ba29-4823ad7da78c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:37.396433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1693b5f-3e12-4e63-ba29-4823ad7da78c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:37.397006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1693b5f-3e12-4e63-ba29-4823ad7da78c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:37.398357 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1693b5f-3e12-4e63-ba29-4823ad7da78c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-d1693b5f-3e12-4e63-ba29-4823ad7da78c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:37.407695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:37.408686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:37.408686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:37.408686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:37.408686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:37.408686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:37.408686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:37.408686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:37.408686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:37.408686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:37.408686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:37.408686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:37.408686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:37.408686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:37.408686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:37.408686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:37.408686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:37.409562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:37.409562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:37.409562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:37.409562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:37.409562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:37.409562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:37.409562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:37.409562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:37.409562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:37.409562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:37.409562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:37.409562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:37.409562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 12:09:37.409562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:37.411836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 512/1010] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:37.414596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186453f3-d886-428b-9bcf-fe44a5ea8e74 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:37.515778 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186453f3-d886-428b-9bcf-fe44a5ea8e74 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:37.517222 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:37.552084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fb9b5-7b8d-46e3-8f33-a2ba25d3dd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:37.552688 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fb9b5-7b8d-46e3-8f33-a2ba25d3dd20 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:37.552962 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fb9b5-7b8d-46e3-8f33-a2ba25d3dd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:37.553227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fb9b5-7b8d-46e3-8f33-a2ba25d3dd20 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:37.553846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fb9b5-7b8d-46e3-8f33-a2ba25d3dd20 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:37.554142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fb9b5-7b8d-46e3-8f33-a2ba25d3dd20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:37.555496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fb9b5-7b8d-46e3-8f33-a2ba25d3dd20 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:37.564758 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 499/1011] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Oct 6 12:09:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:37.565946 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1693b5f-3e12-4e63-ba29-4823ad7da78c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:37.663259 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:37.667327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1693b5f-3e12-4e63-ba29-4823ad7da78c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:37.701313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52a19c-8b4b-4f00-932b-4303cd828716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:37.701905 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52a19c-8b4b-4f00-932b-4303cd828716 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:37.702181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52a19c-8b4b-4f00-932b-4303cd828716 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:37.702447 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52a19c-8b4b-4f00-932b-4303cd828716 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:37.703059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52a19c-8b4b-4f00-932b-4303cd828716 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:37.703358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52a19c-8b4b-4f00-932b-4303cd828716 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:37.704730 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52a19c-8b4b-4f00-932b-4303cd828716 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:37.709593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 513/1012] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Oct 6 12:09:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:37.714965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fb9b5-7b8d-46e3-8f33-a2ba25d3dd20 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:37.813886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fb9b5-7b8d-46e3-8f33-a2ba25d3dd20 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:37.847022 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930459a3-c38f-4bc0-b192-9f702c74bce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:37.847652 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930459a3-c38f-4bc0-b192-9f702c74bce7 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:37.847930 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930459a3-c38f-4bc0-b192-9f702c74bce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:37.848196 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930459a3-c38f-4bc0-b192-9f702c74bce7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:37.849411 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930459a3-c38f-4bc0-b192-9f702c74bce7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:37.850762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930459a3-c38f-4bc0-b192-9f702c74bce7 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:37.875546 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 500/1013] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:09:37 2022] POST /identity/v3/endpoints => generated 334 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:09:38.336815 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706de348-017b-4fad-8e94-8794b345375e None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:38.337347 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706de348-017b-4fad-8e94-8794b345375e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:38.337347 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706de348-017b-4fad-8e94-8794b345375e None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:38.338814 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 514/1014] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:38.343875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a37f3e-9e7f-4f9b-a676-df15d10e46ed None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:38.344169 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a37f3e-9e7f-4f9b-a676-df15d10e46ed None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:38.344446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a37f3e-9e7f-4f9b-a676-df15d10e46ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:38.362358 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-83a37f3e-9e7f-4f9b-a676-df15d10e46ed None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:38.363552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-83a37f3e-9e7f-4f9b-a676-df15d10e46ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:38.380233 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:38.416685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efccb97-baa0-4a9c-b4c6-70534965957f None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:38.417058 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efccb97-baa0-4a9c-b4c6-70534965957f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:38.417227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efccb97-baa0-4a9c-b4c6-70534965957f None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:38.418680 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 515/1015] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:38.423633 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfa53a-0054-4302-89be-a7d09c8f02ba None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:38.423919 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfa53a-0054-4302-89be-a7d09c8f02ba None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:38.424198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfa53a-0054-4302-89be-a7d09c8f02ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:38.440067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 501/1016] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:38 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:38.442500 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fccfa53a-0054-4302-89be-a7d09c8f02ba None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:38.443735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fccfa53a-0054-4302-89be-a7d09c8f02ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:38.515937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 516/1017] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:38 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:38.726152 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a37f3e-9e7f-4f9b-a676-df15d10e46ed None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:38.780279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfa53a-0054-4302-89be-a7d09c8f02ba None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:38.834880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a37f3e-9e7f-4f9b-a676-df15d10e46ed None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:38.875011 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ab1cf-35d8-48fe-b8e2-4172ee614423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:38.875657 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ab1cf-35d8-48fe-b8e2-4172ee614423 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:38.875941 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ab1cf-35d8-48fe-b8e2-4172ee614423 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:38.876216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ab1cf-35d8-48fe-b8e2-4172ee614423 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:38.876866 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ab1cf-35d8-48fe-b8e2-4172ee614423 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:38.878237 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ab1cf-35d8-48fe-b8e2-4172ee614423 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:38.878412 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:38.891076 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfa53a-0054-4302-89be-a7d09c8f02ba None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:38.899555 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 502/1018] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Oct 6 12:09:38 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:09:38.951198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4f1d4-44f3-4949-aa24-bd74549ce0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:38.951929 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4f1d4-44f3-4949-aa24-bd74549ce0b4 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:38.952214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4f1d4-44f3-4949-aa24-bd74549ce0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:38.952543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4f1d4-44f3-4949-aa24-bd74549ce0b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:38.953731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b4f1d4-44f3-4949-aa24-bd74549ce0b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:38.955203 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b4f1d4-44f3-4949-aa24-bd74549ce0b4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:38.959563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 517/1019] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:38.963916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ab1cf-35d8-48fe-b8e2-4172ee614423 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:38.976844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:39.078893 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ab1cf-35d8-48fe-b8e2-4172ee614423 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:39.113908 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c1e95-f6d6-4fa1-b3b3-c85c8ca0c936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:39.114820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c1e95-f6d6-4fa1-b3b3-c85c8ca0c936 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:39.115336 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c1e95-f6d6-4fa1-b3b3-c85c8ca0c936 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:39.115661 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c1e95-f6d6-4fa1-b3b3-c85c8ca0c936 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:39.116377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c1e95-f6d6-4fa1-b3b3-c85c8ca0c936 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:39.117883 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c1e95-f6d6-4fa1-b3b3-c85c8ca0c936 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:39.149040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 503/1020] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Oct 6 12:09:38 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:09:40.129121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430941e-d271-4fdc-8684-0b27d8aaccbd None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:40.129673 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430941e-d271-4fdc-8684-0b27d8aaccbd None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:40.129752 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430941e-d271-4fdc-8684-0b27d8aaccbd None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:40.131087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 518/1021] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:40.136208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e956a-d204-4179-b63e-8febc98f5627 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:40.136616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e956a-d204-4179-b63e-8febc98f5627 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:40.136799 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e956a-d204-4179-b63e-8febc98f5627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:40.155742 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-244e956a-d204-4179-b63e-8febc98f5627 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:40.156936 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-244e956a-d204-4179-b63e-8febc98f5627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:40.214613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:40.228410 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 504/1022] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:40.496528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430941e-d271-4fdc-8684-0b27d8aaccbd None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:40.604267 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430941e-d271-4fdc-8684-0b27d8aaccbd None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:40.638474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021cbca0-ffbf-41b5-b9ad-225a2c4a880f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:40.639107 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cbca0-ffbf-41b5-b9ad-225a2c4a880f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:40.639390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cbca0-ffbf-41b5-b9ad-225a2c4a880f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:40.639696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cbca0-ffbf-41b5-b9ad-225a2c4a880f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:40.650297 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021cbca0-ffbf-41b5-b9ad-225a2c4a880f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:40.655398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-021cbca0-ffbf-41b5-b9ad-225a2c4a880f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 12:09:40.655398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:40.655398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:40.655398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:40.655398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:40.655398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:40.655398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:40.655398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:40.655398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:40.655398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:40.655398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:40.655398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:40.655398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:40.655398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:40.655398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:40.655398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:40.656396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:40.656396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:40.656396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:40.656396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:40.656396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:40.656396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:40.656396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:40.656396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:40.656396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:40.656396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:40.656396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:40.656396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:40.656396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:40.656396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:40.656396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:40.656396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:40.657272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:40.657272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:40.657272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:40.657272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:40.657272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:40.657272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:40.657272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:40.657272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:40.657272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:40.657272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:40.657272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 12:09:40.657272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:40.658590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 519/1023] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Oct 6 12:09:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:40.663013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e956a-d204-4179-b63e-8febc98f5627 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:40.763926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e956a-d204-4179-b63e-8febc98f5627 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:40.797516 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d49a3e-09be-46b6-b055-3ed55bde1a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:40.798127 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d49a3e-09be-46b6-b055-3ed55bde1a56 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:40.798407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d49a3e-09be-46b6-b055-3ed55bde1a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:40.798678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d49a3e-09be-46b6-b055-3ed55bde1a56 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:40.799342 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d49a3e-09be-46b6-b055-3ed55bde1a56 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:40.799753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d49a3e-09be-46b6-b055-3ed55bde1a56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:40.801132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d49a3e-09be-46b6-b055-3ed55bde1a56 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:40.814607 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 505/1024] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Oct 6 12:09:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:40.815919 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021cbca0-ffbf-41b5-b9ad-225a2c4a880f None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:40.837009 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:40.922520 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021cbca0-ffbf-41b5-b9ad-225a2c4a880f None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:40.963272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16413f-4421-42b3-ab3c-14131890d298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:40.963957 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16413f-4421-42b3-ab3c-14131890d298 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:40.964241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16413f-4421-42b3-ab3c-14131890d298 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:40.964517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16413f-4421-42b3-ab3c-14131890d298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:40.976335 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16413f-4421-42b3-ab3c-14131890d298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:40.977821 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16413f-4421-42b3-ab3c-14131890d298 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:40.994275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-2f16413f-4421-42b3-ab3c-14131890d298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:40.994275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:40.994275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:40.994275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:40.994275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:40.994275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:40.994275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:40.994275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:40.994275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:40.994275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:40.994275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:40.994275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:40.994275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:40.994275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:40.994275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:40.994275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:40.995320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:40.996617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:40.996617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:40.996617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:40.999665 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 520/1025] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:41.001177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d49a3e-09be-46b6-b055-3ed55bde1a56 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:41.056958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:41.071656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2507309-3956-4908-8be8-bfc6caabce9d None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:41.071945 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2507309-3956-4908-8be8-bfc6caabce9d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:41.072222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2507309-3956-4908-8be8-bfc6caabce9d None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:41.075554 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 521/1026] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:41.078570 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bfb5b-ce1f-4d23-8d2d-021d71d8ac30 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:41.078852 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bfb5b-ce1f-4d23-8d2d-021d71d8ac30 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:41.079123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bfb5b-ce1f-4d23-8d2d-021d71d8ac30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:41.097503 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b26bfb5b-ce1f-4d23-8d2d-021d71d8ac30 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:41.099226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b26bfb5b-ce1f-4d23-8d2d-021d71d8ac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:41.116304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d49a3e-09be-46b6-b055-3ed55bde1a56 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:41.152059 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da70b2-cf92-48f3-b280-07bc982b833f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:41.153562 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da70b2-cf92-48f3-b280-07bc982b833f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:41.154187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da70b2-cf92-48f3-b280-07bc982b833f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:41.154470 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da70b2-cf92-48f3-b280-07bc982b833f None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:41.155110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da70b2-cf92-48f3-b280-07bc982b833f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:41.155416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da70b2-cf92-48f3-b280-07bc982b833f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:41.156977 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da70b2-cf92-48f3-b280-07bc982b833f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:41.164313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:41.176325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 506/1027] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:41.176515 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 522/1028] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:41.178096 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da70b2-cf92-48f3-b280-07bc982b833f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:41.285504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da70b2-cf92-48f3-b280-07bc982b833f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:41.299359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:41.322175 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25fb5a-6ad4-4873-a096-22ca037d6fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:41.322725 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25fb5a-6ad4-4873-a096-22ca037d6fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:41.323005 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25fb5a-6ad4-4873-a096-22ca037d6fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:41.323276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25fb5a-6ad4-4873-a096-22ca037d6fb8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:41.330553 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25fb5a-6ad4-4873-a096-22ca037d6fb8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:41.332036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25fb5a-6ad4-4873-a096-22ca037d6fb8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:41.334007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba771f-72e0-40b1-8d6e-1d44659f089a None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:41.334294 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba771f-72e0-40b1-8d6e-1d44659f089a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:41.334566 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba771f-72e0-40b1-8d6e-1d44659f089a None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:41.336225 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 523/1029] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:41.340867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f23588-5661-4bdb-8bd9-84a006ae59b8 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:41.341150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f23588-5661-4bdb-8bd9-84a006ae59b8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:41.341423 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f23588-5661-4bdb-8bd9-84a006ae59b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:41.345757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-df25fb5a-6ad4-4873-a096-22ca037d6fb8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:09:41.345757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:41.345757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:41.345757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:41.345757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:41.345757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:41.345757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:41.345757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:41.345757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:41.345757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:41.345757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:41.345757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:41.345757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:09:41.345757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:09:41.345757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:09:41.345757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:41.346707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:41.347784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:41.347784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:41.347784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:09:41.347784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:09:41.347784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:09:41.347784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:09:41.347784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:09:41.347784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:09:41.347784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:09:41.347784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:41.347784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 507/1030] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Oct 6 12:09:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:41.360687 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25fb5a-6ad4-4873-a096-22ca037d6fb8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:41.366250 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-59f23588-5661-4bdb-8bd9-84a006ae59b8 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:41.367432 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-59f23588-5661-4bdb-8bd9-84a006ae59b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:41.443558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 524/1031] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:41.443558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f23588-5661-4bdb-8bd9-84a006ae59b8 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:41.465627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25fb5a-6ad4-4873-a096-22ca037d6fb8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:41.500949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce56b9-bf57-4405-8772-b9b230d76d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:41.501842 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce56b9-bf57-4405-8772-b9b230d76d03 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:41.502303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce56b9-bf57-4405-8772-b9b230d76d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:41.502757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce56b9-bf57-4405-8772-b9b230d76d03 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:41.503678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce56b9-bf57-4405-8772-b9b230d76d03 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:41.504192 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce56b9-bf57-4405-8772-b9b230d76d03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:41.506123 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce56b9-bf57-4405-8772-b9b230d76d03 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:41.528158 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 508/1032] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Oct 6 12:09:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:41.529418 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce56b9-bf57-4405-8772-b9b230d76d03 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:41.548551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f23588-5661-4bdb-8bd9-84a006ae59b8 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:41.560887 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:41.578147 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:41.586965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c19779-c946-46cb-99ff-69849097c2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:41.587638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c19779-c946-46cb-99ff-69849097c2da None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:41.587920 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c19779-c946-46cb-99ff-69849097c2da None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:41.588190 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c19779-c946-46cb-99ff-69849097c2da None admin] PATH_INFO: `/v3/services/image` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:41.588773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c19779-c946-46cb-99ff-69849097c2da None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:41.590574 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c19779-c946-46cb-99ff-69849097c2da None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-53c19779-c946-46cb-99ff-69849097c2da None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:41.599324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:41.600346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:41.600346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:41.600346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:41.600346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:41.600346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:41.600346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:41.600346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:41.600346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:41.600346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:41.600346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:41.600346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:41.600346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:41.600346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:41.600346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:41.600346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:41.600346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:41.601248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:41.601248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:41.601248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:41.601248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:41.601248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:41.601248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:41.601248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:41.601248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:41.601248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:41.601248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:41.601248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:41.601248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:41.601248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 12:09:41.601248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:41.601248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 525/1033] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Oct 6 12:09:41 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:41.606129 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c19779-c946-46cb-99ff-69849097c2da None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:41.642942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce56b9-bf57-4405-8772-b9b230d76d03 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:41.680260 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534d82c-1580-4a9b-866e-62026407d59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:41.680872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534d82c-1580-4a9b-866e-62026407d59a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:41.681154 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534d82c-1580-4a9b-866e-62026407d59a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:41.681422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534d82c-1580-4a9b-866e-62026407d59a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:41.716272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c19779-c946-46cb-99ff-69849097c2da None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:41.748468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534d82c-1580-4a9b-866e-62026407d59a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:41.749513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d52b0-5bc1-45a7-95f5-bd39b6ee830b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:41.749911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534d82c-1580-4a9b-866e-62026407d59a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:41.750115 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d52b0-5bc1-45a7-95f5-bd39b6ee830b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:41.750396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d52b0-5bc1-45a7-95f5-bd39b6ee830b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:41.750670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d52b0-5bc1-45a7-95f5-bd39b6ee830b None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:41.751289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d52b0-5bc1-45a7-95f5-bd39b6ee830b None admin] RBAC: Adding query filter params (name=image) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:41.751629 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d52b0-5bc1-45a7-95f5-bd39b6ee830b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:41.752973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d52b0-5bc1-45a7-95f5-bd39b6ee830b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:41.757805 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 526/1034] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:41.758576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d52b0-5bc1-45a7-95f5-bd39b6ee830b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:41.777792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:41.815073 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:41.848883 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 509/1035] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:09:41 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 06 12:09:41.849645 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534d82c-1580-4a9b-866e-62026407d59a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:41.861328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d52b0-5bc1-45a7-95f5-bd39b6ee830b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:41.893358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9702d9-00ff-4e4f-86cf-9848e191a4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:41.893971 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9702d9-00ff-4e4f-86cf-9848e191a4dd None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:41.894243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9702d9-00ff-4e4f-86cf-9848e191a4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:41.894511 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9702d9-00ff-4e4f-86cf-9848e191a4dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:41.896142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9702d9-00ff-4e4f-86cf-9848e191a4dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:41.897556 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9702d9-00ff-4e4f-86cf-9848e191a4dd None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:41.899101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 527/1036] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:41.899873 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9702d9-00ff-4e4f-86cf-9848e191a4dd None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:41.945834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534d82c-1580-4a9b-866e-62026407d59a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:41.974755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:41.981147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea26ac8-0e42-414e-ac10-f143c15eb9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:41.981820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea26ac8-0e42-414e-ac10-f143c15eb9dd None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:41.982169 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea26ac8-0e42-414e-ac10-f143c15eb9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:41.982450 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea26ac8-0e42-414e-ac10-f143c15eb9dd None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:41.983062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea26ac8-0e42-414e-ac10-f143c15eb9dd None admin] RBAC: Adding query filter params (type=image) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:41.983361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea26ac8-0e42-414e-ac10-f143c15eb9dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:41.984738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea26ac8-0e42-414e-ac10-f143c15eb9dd None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:41.989871 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 510/1037] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:41.990603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea26ac8-0e42-414e-ac10-f143c15eb9dd None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:42.001318 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9702d9-00ff-4e4f-86cf-9848e191a4dd None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:42.026169 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:42.035099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b741d7-d740-4f8a-b313-6cdfe0d8851a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:42.035813 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b741d7-d740-4f8a-b313-6cdfe0d8851a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:42.036094 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b741d7-d740-4f8a-b313-6cdfe0d8851a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:42.036367 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b741d7-d740-4f8a-b313-6cdfe0d8851a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:42.046490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b741d7-d740-4f8a-b313-6cdfe0d8851a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:42.047934 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b741d7-d740-4f8a-b313-6cdfe0d8851a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:42.062221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-46b741d7-d740-4f8a-b313-6cdfe0d8851a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:42.062221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:42.062221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:42.062221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:42.062221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:42.062221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:42.062221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:42.062221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:42.062221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:42.062221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:42.062221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:42.062221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:42.062221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:42.062221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:42.062221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:42.062221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:42.063287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:42.064418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:42.064418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:42.064418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:42.064418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 528/1038] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:42.064418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b741d7-d740-4f8a-b313-6cdfe0d8851a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:42.096346 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea26ac8-0e42-414e-ac10-f143c15eb9dd None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:42.128794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f46c0-e956-443c-a2bf-989cfa9923cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:42.129455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f46c0-e956-443c-a2bf-989cfa9923cf None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:42.129740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f46c0-e956-443c-a2bf-989cfa9923cf None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:42.130015 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f46c0-e956-443c-a2bf-989cfa9923cf None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:42.130655 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f46c0-e956-443c-a2bf-989cfa9923cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:42.132129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f46c0-e956-443c-a2bf-989cfa9923cf None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:42.154243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 511/1039] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:09:41 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:09:42.155077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f46c0-e956-443c-a2bf-989cfa9923cf None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:42.169122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b741d7-d740-4f8a-b313-6cdfe0d8851a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:42.209827 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b5be0-8816-426c-8234-a70ec63f0af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:42.210472 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b5be0-8816-426c-8234-a70ec63f0af3 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:42.210758 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b5be0-8816-426c-8234-a70ec63f0af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:42.211035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b5be0-8816-426c-8234-a70ec63f0af3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:42.211795 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b5be0-8816-426c-8234-a70ec63f0af3 None admin] RBAC: Adding query filter params (interface=public, service_id=b515799e14d64fa781184edf610c0d04, region_id=RegionOne) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:42.212115 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b5be0-8816-426c-8234-a70ec63f0af3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:42.213361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:42.213514 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b5be0-8816-426c-8234-a70ec63f0af3 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:42.220804 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 529/1040] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Oct 6 12:09:42 2022] GET /identity/v3/endpoints?service_id=b515799e14d64fa781184edf610c0d04&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:42.227282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b5be0-8816-426c-8234-a70ec63f0af3 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:42.277727 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f46c0-e956-443c-a2bf-989cfa9923cf None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:42.285301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:42.315642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b72d2-22e9-469f-9c00-9f63295a6781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:42.316275 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b72d2-22e9-469f-9c00-9f63295a6781 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:42.316552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b72d2-22e9-469f-9c00-9f63295a6781 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:42.316858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b72d2-22e9-469f-9c00-9f63295a6781 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:42.317509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b72d2-22e9-469f-9c00-9f63295a6781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:42.317814 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b72d2-22e9-469f-9c00-9f63295a6781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:42.319193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b72d2-22e9-469f-9c00-9f63295a6781 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:42.331431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 512/1041] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:42.337180 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b72d2-22e9-469f-9c00-9f63295a6781 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:42.355952 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b5be0-8816-426c-8234-a70ec63f0af3 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:42.397757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9229a69-e291-41b9-8118-ef91d5636ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:42.398192 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9229a69-e291-41b9-8118-ef91d5636ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:42.398854 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9229a69-e291-41b9-8118-ef91d5636ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:42.398854 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9229a69-e291-41b9-8118-ef91d5636ca9 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:42.399481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9229a69-e291-41b9-8118-ef91d5636ca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:42.400963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9229a69-e291-41b9-8118-ef91d5636ca9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:42.408902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 530/1042] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Oct 6 12:09:42 2022] GET /identity/v3/services => generated 1957 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:09:42.452024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b72d2-22e9-469f-9c00-9f63295a6781 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:42.464844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:42.492256 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7e3dc-ef22-499d-bcd5-853ecdbb4ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:42.493278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7e3dc-ef22-499d-bcd5-853ecdbb4ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:42.493755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7e3dc-ef22-499d-bcd5-853ecdbb4ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:42.494239 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7e3dc-ef22-499d-bcd5-853ecdbb4ac1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:42.502159 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7e3dc-ef22-499d-bcd5-853ecdbb4ac1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:42.504076 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7e3dc-ef22-499d-bcd5-853ecdbb4ac1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:42.516616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-ace7e3dc-ef22-499d-bcd5-853ecdbb4ac1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:42.516616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:42.516616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:42.516616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:42.516616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:42.516616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:42.516616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:42.516616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:42.516616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:42.516616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:42.516616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:42.516616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:42.516616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:09:42.516616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:09:42.516616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:42.517600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:42.518539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:09:42.518539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:09:42.518539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:09:42.518539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:09:42.518539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:09:42.518539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:09:42.518539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:42.518539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:42.521102 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 513/1043] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:42.527402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9229a69-e291-41b9-8118-ef91d5636ca9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:42.636534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9229a69-e291-41b9-8118-ef91d5636ca9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:42.644034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:42.672957 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20f54e-ea72-4d05-8de2-b1eddf138924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:42.673487 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20f54e-ea72-4d05-8de2-b1eddf138924 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:42.673766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20f54e-ea72-4d05-8de2-b1eddf138924 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:42.674039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20f54e-ea72-4d05-8de2-b1eddf138924 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:42.674723 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20f54e-ea72-4d05-8de2-b1eddf138924 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:42.675028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20f54e-ea72-4d05-8de2-b1eddf138924 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:42.676517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20f54e-ea72-4d05-8de2-b1eddf138924 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:42.694600 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 531/1044] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:09:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:42.695840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7e3dc-ef22-499d-bcd5-853ecdbb4ac1 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:42.798713 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7e3dc-ef22-499d-bcd5-853ecdbb4ac1 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:42.831623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bad904-baef-49cf-a1ce-493a22935376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:42.832226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bad904-baef-49cf-a1ce-493a22935376 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:42.833020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bad904-baef-49cf-a1ce-493a22935376 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:42.833293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bad904-baef-49cf-a1ce-493a22935376 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:42.834004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bad904-baef-49cf-a1ce-493a22935376 None admin] RBAC: Adding query filter params (role.id=14d78f802e8d43d684535749f98404c7, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=5a46e63d34564d34b6b5752706375798) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:42.834309 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bad904-baef-49cf-a1ce-493a22935376 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:42.835758 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bad904-baef-49cf-a1ce-493a22935376 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:42.847531 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 514/1045] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:09:42 2022] GET /identity/v3/role_assignments?user.id=5a46e63d34564d34b6b5752706375798&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=14d78f802e8d43d684535749f98404c7 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:44.372727 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804897f-d5d3-4d29-96f0-9f7178438672 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:44.373262 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804897f-d5d3-4d29-96f0-9f7178438672 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:44.373262 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804897f-d5d3-4d29-96f0-9f7178438672 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:44.374637 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 532/1046] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:44.380434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d835d-ec59-4f75-9f7e-a6371b220db9 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:44.380842 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d835d-ec59-4f75-9f7e-a6371b220db9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:44.381006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d835d-ec59-4f75-9f7e-a6371b220db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:44.399195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2b6d835d-ec59-4f75-9f7e-a6371b220db9 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:44.400398 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2b6d835d-ec59-4f75-9f7e-a6371b220db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:44.427343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:44.471102 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 515/1047] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:44.577573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27beb8-5caf-45f1-ac4e-3950ac74537c None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:44.577906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27beb8-5caf-45f1-ac4e-3950ac74537c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:44.578085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27beb8-5caf-45f1-ac4e-3950ac74537c None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:44.579433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 533/1048] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:44.585232 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3160b-37e6-45fa-9b06-4378ad3a3926 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:44.585525 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3160b-37e6-45fa-9b06-4378ad3a3926 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:44.585802 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3160b-37e6-45fa-9b06-4378ad3a3926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:44.603918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-30c3160b-37e6-45fa-9b06-4378ad3a3926 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:44.605094 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-30c3160b-37e6-45fa-9b06-4378ad3a3926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:44.671123 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 516/1049] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:44.736412 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27beb8-5caf-45f1-ac4e-3950ac74537c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:44.836546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27beb8-5caf-45f1-ac4e-3950ac74537c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:44.876235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289107a3-ffd5-44e9-93f8-b5966ab147bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:44.876869 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289107a3-ffd5-44e9-93f8-b5966ab147bc None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:44.877145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289107a3-ffd5-44e9-93f8-b5966ab147bc None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:44.877411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289107a3-ffd5-44e9-93f8-b5966ab147bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:44.887739 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289107a3-ffd5-44e9-93f8-b5966ab147bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:44.892695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-289107a3-ffd5-44e9-93f8-b5966ab147bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:09:44.892695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:44.892695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:44.892695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:44.892695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:44.892695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:44.892695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:44.892695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:44.892695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:44.892695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:44.892695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:44.892695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:44.892695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:44.892695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:44.892695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:44.892695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:44.893667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:44.893667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:44.893667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:44.893667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:44.893667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:44.893667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:44.893667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:44.893667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:44.893667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:44.893667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:44.893667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:44.893667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:44.893667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:44.893667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:44.893667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:44.893667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:44.894541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:44.894541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:44.894541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:44.894541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:44.894541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:44.894541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:44.894541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:44.894541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:44.894541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:44.894541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:44.894541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:09:44.894541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:44.899207 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 534/1050] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:09:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:44.900610 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3160b-37e6-45fa-9b06-4378ad3a3926 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:44.942919 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289107a3-ffd5-44e9-93f8-b5966ab147bc None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:44.977044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:45.008669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:45.015745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3160b-37e6-45fa-9b06-4378ad3a3926 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:45.049016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1e3ba-935a-4526-9003-bbab21aea3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:45.049622 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1e3ba-935a-4526-9003-bbab21aea3d8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:45.049898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1e3ba-935a-4526-9003-bbab21aea3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:45.050164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1e3ba-935a-4526-9003-bbab21aea3d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:45.050288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289107a3-ffd5-44e9-93f8-b5966ab147bc None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:45.050819 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1e3ba-935a-4526-9003-bbab21aea3d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:45.051662 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1e3ba-935a-4526-9003-bbab21aea3d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:45.053015 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1e3ba-935a-4526-9003-bbab21aea3d8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:45.060807 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 517/1051] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:09:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:45.062015 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4da5b7-4611-47be-a852-76c8ada4390c None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:45.062302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4da5b7-4611-47be-a852-76c8ada4390c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:45.062572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4da5b7-4611-47be-a852-76c8ada4390c None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:45.064096 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 518/1052] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:45.068479 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4da5b7-4611-47be-a852-76c8ada4390c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:45.088292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d12df-7114-40fc-badf-4c1522804861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:45.088872 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d12df-7114-40fc-badf-4c1522804861 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:45.089148 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d12df-7114-40fc-badf-4c1522804861 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:45.089413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d12df-7114-40fc-badf-4c1522804861 None admin] PATH_INFO: `/v3/services/image` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:45.089987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d12df-7114-40fc-badf-4c1522804861 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:45.091362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d12df-7114-40fc-badf-4c1522804861 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-069d12df-7114-40fc-badf-4c1522804861 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:45.100575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:45.101605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:45.101605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:45.101605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:45.101605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:45.101605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:45.101605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:45.101605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:09:45.101605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:09:45.101605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:09:45.101605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:09:45.101605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:45.101605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:45.101605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:45.101605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:45.101605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:45.101605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:45.102490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:45.102490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:45.102490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:45.102490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:45.102490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:45.102490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:45.102490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:45.102490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:45.102490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:45.102490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:45.102490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:09:45.102490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:09:45.102490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 12:09:45.102490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:45.102490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 535/1053] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Oct 6 12:09:44 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:45.103627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd42a5-c367-4151-923b-c9a8673e4107 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:45.103627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd42a5-c367-4151-923b-c9a8673e4107 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:45.103627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd42a5-c367-4151-923b-c9a8673e4107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:45.120543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-17fd42a5-c367-4151-923b-c9a8673e4107 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:45.121717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-17fd42a5-c367-4151-923b-c9a8673e4107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:45.172466 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4da5b7-4611-47be-a852-76c8ada4390c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:45.187600 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 536/1054] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:45.188444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd42a5-c367-4151-923b-c9a8673e4107 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:45.210101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:45.217163 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17b0da-b346-469d-97cc-82bd9d7b415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:45.217794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17b0da-b346-469d-97cc-82bd9d7b415e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:45.218095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17b0da-b346-469d-97cc-82bd9d7b415e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:45.218401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17b0da-b346-469d-97cc-82bd9d7b415e None admin] PATH_INFO: `/v3/users/a1d4ea31b85b484c9dba0e773de9797b` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:45.232430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17b0da-b346-469d-97cc-82bd9d7b415e None admin] RBAC: Authorizing `identity:get_user(user_id=a1d4ea31b85b484c9dba0e773de9797b)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:45.233862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17b0da-b346-469d-97cc-82bd9d7b415e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:45.241016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 519/1055] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:09:45 2022] GET /identity/v3/users/a1d4ea31b85b484c9dba0e773de9797b => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:45.242680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17b0da-b346-469d-97cc-82bd9d7b415e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:45.260856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:45.300011 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd42a5-c367-4151-923b-c9a8673e4107 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:45.335662 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6bf5a-8105-4c64-be1e-6f8008014cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:45.336386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6bf5a-8105-4c64-be1e-6f8008014cab None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:45.336668 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6bf5a-8105-4c64-be1e-6f8008014cab None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:45.336938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6bf5a-8105-4c64-be1e-6f8008014cab None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:45.337561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6bf5a-8105-4c64-be1e-6f8008014cab None admin] RBAC: Adding query filter params (name=image) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:45.337861 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6bf5a-8105-4c64-be1e-6f8008014cab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:45.339567 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6bf5a-8105-4c64-be1e-6f8008014cab None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:45.346245 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 537/1056] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:45.349051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17b0da-b346-469d-97cc-82bd9d7b415e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:45.352170 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6bf5a-8105-4c64-be1e-6f8008014cab None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:45.384496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90cfe9-2623-4c11-9c6a-44aad361694b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:45.385109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90cfe9-2623-4c11-9c6a-44aad361694b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:45.385390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90cfe9-2623-4c11-9c6a-44aad361694b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:45.385661 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90cfe9-2623-4c11-9c6a-44aad361694b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:45.440163 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:45.452415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90cfe9-2623-4c11-9c6a-44aad361694b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:45.453953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90cfe9-2623-4c11-9c6a-44aad361694b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:45.457719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6bf5a-8105-4c64-be1e-6f8008014cab None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:45.468512 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:45.491918 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbf082-89df-4706-b459-91f33ec16106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:45.492518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbf082-89df-4706-b459-91f33ec16106 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:45.492799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbf082-89df-4706-b459-91f33ec16106 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:45.493069 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbf082-89df-4706-b459-91f33ec16106 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:45.493691 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbf082-89df-4706-b459-91f33ec16106 None admin] RBAC: Adding query filter params (type=image) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:45.493992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbf082-89df-4706-b459-91f33ec16106 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:45.495354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbf082-89df-4706-b459-91f33ec16106 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:45.500608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 538/1057] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:45.501762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbf082-89df-4706-b459-91f33ec16106 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:45.553258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 520/1058] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:09:45 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:09:45.554070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90cfe9-2623-4c11-9c6a-44aad361694b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:45.604181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbf082-89df-4706-b459-91f33ec16106 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:45.637065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84720a6-3e2f-447f-a4fe-29602b5e0a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:45.637684 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84720a6-3e2f-447f-a4fe-29602b5e0a35 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:45.637965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84720a6-3e2f-447f-a4fe-29602b5e0a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:45.638250 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84720a6-3e2f-447f-a4fe-29602b5e0a35 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:45.648697 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90cfe9-2623-4c11-9c6a-44aad361694b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:45.649413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84720a6-3e2f-447f-a4fe-29602b5e0a35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:45.654547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-b84720a6-3e2f-447f-a4fe-29602b5e0a35 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 12:09:45.654547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:45.654547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:45.654547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:45.654547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:45.654547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:45.654547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:45.654547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:45.654547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:45.654547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:45.654547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:45.654547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:45.654547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:45.654547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:45.654547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:45.654547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:45.655482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:45.655482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:45.655482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:45.655482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:45.655482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:45.655482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:45.655482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:45.655482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:45.655482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:45.655482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:45.655482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:45.655482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:45.655482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:45.655482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:45.655482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:45.655482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:45.656382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:45.656382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:45.656382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:45.656382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:45.656382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:45.656382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:45.656382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:45.656382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:45.656382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:45.656382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:45.656382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 12:09:45.656382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:45.656382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 539/1059] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Oct 6 12:09:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:45.657089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84720a6-3e2f-447f-a4fe-29602b5e0a35 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:45.665075 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:45.669287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:45.685401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16525ac3-949a-4ab1-b749-70e4b1f4f4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:45.686003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16525ac3-949a-4ab1-b749-70e4b1f4f4ac None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:45.686279 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16525ac3-949a-4ab1-b749-70e4b1f4f4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:45.686552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16525ac3-949a-4ab1-b749-70e4b1f4f4ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:45.687196 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16525ac3-949a-4ab1-b749-70e4b1f4f4ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:45.688591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16525ac3-949a-4ab1-b749-70e4b1f4f4ac None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:45.719132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 521/1060] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:09:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:09:45.719132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16525ac3-949a-4ab1-b749-70e4b1f4f4ac None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:45.783162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84720a6-3e2f-447f-a4fe-29602b5e0a35 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:45.830055 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4368140-dba6-4cc0-bcf3-7079d5651373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:45.830694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4368140-dba6-4cc0-bcf3-7079d5651373 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:45.830965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4368140-dba6-4cc0-bcf3-7079d5651373 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:45.831234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4368140-dba6-4cc0-bcf3-7079d5651373 None admin] PATH_INFO: `/v3/projects/1df28446404b46ee8060b28c7c3da4be` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:45.833067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4368140-dba6-4cc0-bcf3-7079d5651373 None admin] RBAC: Authorizing `identity:get_project(project_id=1df28446404b46ee8060b28c7c3da4be)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:45.834503 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4368140-dba6-4cc0-bcf3-7079d5651373 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:45.835248 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16525ac3-949a-4ab1-b749-70e4b1f4f4ac None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:45.841894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 540/1061] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:09:45 2022] GET /identity/v3/projects/1df28446404b46ee8060b28c7c3da4be => generated 311 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:45.842750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4368140-dba6-4cc0-bcf3-7079d5651373 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:45.881288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66f592-4621-46ca-9ad7-dc48210c00db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:45.881826 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66f592-4621-46ca-9ad7-dc48210c00db None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:45.882102 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66f592-4621-46ca-9ad7-dc48210c00db None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:45.882369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66f592-4621-46ca-9ad7-dc48210c00db None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:45.883036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66f592-4621-46ca-9ad7-dc48210c00db None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:45.883338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66f592-4621-46ca-9ad7-dc48210c00db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:45.884724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66f592-4621-46ca-9ad7-dc48210c00db None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:45.893948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 522/1062] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Oct 6 12:09:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:45.900197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66f592-4621-46ca-9ad7-dc48210c00db None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:45.923890 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:45.933652 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:45.956622 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4368140-dba6-4cc0-bcf3-7079d5651373 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:45.993403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c35d0c-9657-4c76-b6a1-6cd7decdfbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:45.994299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c35d0c-9657-4c76-b6a1-6cd7decdfbcc None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:45.994578 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c35d0c-9657-4c76-b6a1-6cd7decdfbcc None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:45.994850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c35d0c-9657-4c76-b6a1-6cd7decdfbcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:45.995604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c35d0c-9657-4c76-b6a1-6cd7decdfbcc None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=1df28446404b46ee8060b28c7c3da4be, user.id=a1d4ea31b85b484c9dba0e773de9797b) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:45.995915 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c35d0c-9657-4c76-b6a1-6cd7decdfbcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:45.997768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c35d0c-9657-4c76-b6a1-6cd7decdfbcc None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:46.003257 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 541/1063] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:09:45 2022] GET /identity/v3/role_assignments?user.id=a1d4ea31b85b484c9dba0e773de9797b&scope.project.id=1df28446404b46ee8060b28c7c3da4be&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:46.009904 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66f592-4621-46ca-9ad7-dc48210c00db None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:46.048075 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aee745-6eb7-4953-b2df-99cb03fcd096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:46.048720 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aee745-6eb7-4953-b2df-99cb03fcd096 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:46.049000 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aee745-6eb7-4953-b2df-99cb03fcd096 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:46.049270 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aee745-6eb7-4953-b2df-99cb03fcd096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:46.061240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aee745-6eb7-4953-b2df-99cb03fcd096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:46.062720 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aee745-6eb7-4953-b2df-99cb03fcd096 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:46.079201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-e5aee745-6eb7-4953-b2df-99cb03fcd096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:46.079201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:46.079201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:46.079201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:46.079201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:46.079201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:46.079201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:46.079201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:46.079201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:46.079201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:46.079201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:46.079201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:46.079201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:46.079201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:46.079201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:46.079201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:46.080293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:46.081245 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:46.081245 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:46.081245 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:46.083854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 523/1064] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:46.086891 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c35d0c-9657-4c76-b6a1-6cd7decdfbcc None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:46.200558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c35d0c-9657-4c76-b6a1-6cd7decdfbcc None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:46.237166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fff7d-53af-4e37-a6fb-41d9481d400b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:46.237777 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fff7d-53af-4e37-a6fb-41d9481d400b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:46.239590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fff7d-53af-4e37-a6fb-41d9481d400b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:46.239590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fff7d-53af-4e37-a6fb-41d9481d400b None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:46.239590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fff7d-53af-4e37-a6fb-41d9481d400b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:46.239973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fff7d-53af-4e37-a6fb-41d9481d400b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:46.241920 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fff7d-53af-4e37-a6fb-41d9481d400b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:46.254846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:46.262897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aee745-6eb7-4953-b2df-99cb03fcd096 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:46.263351 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 542/1065] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:46.347072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:46.367963 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aee745-6eb7-4953-b2df-99cb03fcd096 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:46.402244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd0509-7b71-4357-8c56-6e41a2ab03fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:46.402898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd0509-7b71-4357-8c56-6e41a2ab03fa None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:46.403138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd0509-7b71-4357-8c56-6e41a2ab03fa None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:46.403403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd0509-7b71-4357-8c56-6e41a2ab03fa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:46.411874 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd0509-7b71-4357-8c56-6e41a2ab03fa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:46.413314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd0509-7b71-4357-8c56-6e41a2ab03fa None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:46.424392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-45dd0509-7b71-4357-8c56-6e41a2ab03fa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:09:46.424392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:46.424392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:46.424392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:46.424392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:46.424392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:46.424392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:46.424392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:46.424392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:46.424392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:46.424392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:46.424392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:46.424392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:09:46.424392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:09:46.424392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:09:46.424392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:46.425388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:46.426934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:46.426934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:46.426934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:09:46.426934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:09:46.426934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:09:46.426934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:09:46.426934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:09:46.426934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:09:46.426934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:09:46.426934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:46.426934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 524/1066] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Oct 6 12:09:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:46.434236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fff7d-53af-4e37-a6fb-41d9481d400b None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:46.538738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fff7d-53af-4e37-a6fb-41d9481d400b None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:46.573291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8b8c3-f0f6-447c-81bb-2933430f0bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:46.573896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8b8c3-f0f6-447c-81bb-2933430f0bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:46.574178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8b8c3-f0f6-447c-81bb-2933430f0bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:46.574455 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8b8c3-f0f6-447c-81bb-2933430f0bb1 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:46.575170 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8b8c3-f0f6-447c-81bb-2933430f0bb1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:46.575478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8b8c3-f0f6-447c-81bb-2933430f0bb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:46.576932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8b8c3-f0f6-447c-81bb-2933430f0bb1 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:46.598326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 543/1067] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Oct 6 12:09:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:46.599541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd0509-7b71-4357-8c56-6e41a2ab03fa None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:46.702182 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd0509-7b71-4357-8c56-6e41a2ab03fa None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:46.735477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8200c-84d6-4c92-9796-de495a8a233d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:46.736133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8200c-84d6-4c92-9796-de495a8a233d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:46.736414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8200c-84d6-4c92-9796-de495a8a233d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:46.736688 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8200c-84d6-4c92-9796-de495a8a233d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:46.745958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:46.748922 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8200c-84d6-4c92-9796-de495a8a233d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:46.750636 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8200c-84d6-4c92-9796-de495a8a233d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:46.764867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-3cd8200c-84d6-4c92-9796-de495a8a233d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:46.764867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:46.764867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:46.764867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:46.764867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:46.764867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:46.764867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:46.764867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:46.764867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:46.764867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:46.764867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:46.764867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:46.764867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:46.764867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:46.764867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:46.764867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:46.765846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:46.766817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:46.766817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:46.766817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:46.766817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 525/1068] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:46.773030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8b8c3-f0f6-447c-81bb-2933430f0bb1 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:46.828698 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:46.876421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8b8c3-f0f6-447c-81bb-2933430f0bb1 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:46.911218 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af8089-0cf3-4bed-8cb1-035b3f808b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:46.911849 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af8089-0cf3-4bed-8cb1-035b3f808b9e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:46.912131 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af8089-0cf3-4bed-8cb1-035b3f808b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:46.912405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af8089-0cf3-4bed-8cb1-035b3f808b9e None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:46.913041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af8089-0cf3-4bed-8cb1-035b3f808b9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:46.913346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af8089-0cf3-4bed-8cb1-035b3f808b9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:46.914700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af8089-0cf3-4bed-8cb1-035b3f808b9e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:46.930829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 544/1069] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:46.932092 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8200c-84d6-4c92-9796-de495a8a233d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:47.041276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8200c-84d6-4c92-9796-de495a8a233d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:47.074531 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f059ed8-a86e-4281-91c9-60197b7f8a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:47.075165 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f059ed8-a86e-4281-91c9-60197b7f8a49 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:47.075448 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f059ed8-a86e-4281-91c9-60197b7f8a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:47.075751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f059ed8-a86e-4281-91c9-60197b7f8a49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:47.082364 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f059ed8-a86e-4281-91c9-60197b7f8a49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:47.083768 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f059ed8-a86e-4281-91c9-60197b7f8a49 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:47.095499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-4f059ed8-a86e-4281-91c9-60197b7f8a49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:47.095499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:47.095499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:47.095499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:47.095499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:47.095499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:47.095499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:47.095499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:47.095499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:47.095499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:47.095499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:47.095499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:47.095499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:09:47.095499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:09:47.095499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:47.096429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:47.097383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:09:47.097383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:09:47.097383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:09:47.097383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:09:47.097383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:09:47.097383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:09:47.097383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:47.097383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:47.097383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 526/1070] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:47.104178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af8089-0cf3-4bed-8cb1-035b3f808b9e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:47.206490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af8089-0cf3-4bed-8cb1-035b3f808b9e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:47.220487 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:47.247280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430bf688-f2f7-4d81-9e0e-be46792e983f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:47.250624 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bf688-f2f7-4d81-9e0e-be46792e983f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:47.250624 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bf688-f2f7-4d81-9e0e-be46792e983f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:47.250624 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bf688-f2f7-4d81-9e0e-be46792e983f None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:47.250624 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430bf688-f2f7-4d81-9e0e-be46792e983f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:47.250624 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430bf688-f2f7-4d81-9e0e-be46792e983f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:47.251358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430bf688-f2f7-4d81-9e0e-be46792e983f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:47.268912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 545/1071] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:09:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:47.270617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f059ed8-a86e-4281-91c9-60197b7f8a49 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:47.325316 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:47.374790 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f059ed8-a86e-4281-91c9-60197b7f8a49 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:47.408218 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463ca93-9320-4d6f-a40b-8d1276b7a43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:47.408928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463ca93-9320-4d6f-a40b-8d1276b7a43a None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:47.409155 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463ca93-9320-4d6f-a40b-8d1276b7a43a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:47.409429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463ca93-9320-4d6f-a40b-8d1276b7a43a None admin] PATH_INFO: `/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/5a46e63d34564d34b6b5752706375798/roles/14d78f802e8d43d684535749f98404c7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:47.412147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463ca93-9320-4d6f-a40b-8d1276b7a43a None admin] RBAC: Authorizing `identity:create_grant(project_id=314d3d13ba2e4a0f93f5d3f34c50a96f, user_id=5a46e63d34564d34b6b5752706375798, role_id=14d78f802e8d43d684535749f98404c7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:47.413569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463ca93-9320-4d6f-a40b-8d1276b7a43a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:47.426657 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 527/1072] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:09:47 2022] PUT /identity/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/5a46e63d34564d34b6b5752706375798/roles/14d78f802e8d43d684535749f98404c7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:09:47.889719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6182bf-ffac-40da-8f97-27531dd0f022 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:47.890273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6182bf-ffac-40da-8f97-27531dd0f022 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:47.890273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6182bf-ffac-40da-8f97-27531dd0f022 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:47.891630 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 546/1073] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:47.897383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef206b-0459-4513-9692-4d22fcd04d75 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:47.897666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef206b-0459-4513-9692-4d22fcd04d75 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:47.897934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef206b-0459-4513-9692-4d22fcd04d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:47.916710 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-76ef206b-0459-4513-9692-4d22fcd04d75 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:47.917881 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-76ef206b-0459-4513-9692-4d22fcd04d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:47.988947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 528/1074] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:48.148258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4c725-6b26-4c67-8f95-6c44f14a380c None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:48.148446 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4c725-6b26-4c67-8f95-6c44f14a380c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:48.148735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4c725-6b26-4c67-8f95-6c44f14a380c None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:48.150219 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 547/1075] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:48.155444 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda47ff8-481b-4238-8891-2dc727e0db63 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:48.155762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda47ff8-481b-4238-8891-2dc727e0db63 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:48.156036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda47ff8-481b-4238-8891-2dc727e0db63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:48.174483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-bda47ff8-481b-4238-8891-2dc727e0db63 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:48.176246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bda47ff8-481b-4238-8891-2dc727e0db63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:48.212661 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:48.259551 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 529/1076] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:48.259724 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4c725-6b26-4c67-8f95-6c44f14a380c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:48.363232 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4c725-6b26-4c67-8f95-6c44f14a380c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:48.397309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fcd75-238c-46be-8915-e465f8f525c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:48.398177 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fcd75-238c-46be-8915-e465f8f525c7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:48.398717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fcd75-238c-46be-8915-e465f8f525c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:48.399184 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fcd75-238c-46be-8915-e465f8f525c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:48.440201 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:48.467433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fcd75-238c-46be-8915-e465f8f525c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:48.468909 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fcd75-238c-46be-8915-e465f8f525c7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:48.526057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda47ff8-481b-4238-8891-2dc727e0db63 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:48.571855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 548/1077] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:09:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 06 12:09:48.574948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fcd75-238c-46be-8915-e465f8f525c7 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:48.632356 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda47ff8-481b-4238-8891-2dc727e0db63 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:48.650807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:48.667057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a90cd-393b-4936-90f6-e450f53cb47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:48.667736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a90cd-393b-4936-90f6-e450f53cb47d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:48.668020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a90cd-393b-4936-90f6-e450f53cb47d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:48.668287 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a90cd-393b-4936-90f6-e450f53cb47d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:48.679221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a90cd-393b-4936-90f6-e450f53cb47d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:48.679445 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fcd75-238c-46be-8915-e465f8f525c7 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:48.684113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-405a90cd-393b-4936-90f6-e450f53cb47d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:09:48.684113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:48.684113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:48.684113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:48.684113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:48.684113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:48.684113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:48.684113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:48.684113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:48.684113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:48.684113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:48.684113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:48.684113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:48.684113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:48.684113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:48.684113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:48.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:48.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:48.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:48.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:48.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:48.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:48.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:48.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:48.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:48.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:48.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:48.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:48.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:48.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:48.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:48.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:48.685982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:48.685982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:48.685982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:48.685982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:48.685982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:48.685982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:48.685982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:48.685982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:48.685982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:48.685982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:48.685982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:09:48.685982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:48.685982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 530/1078] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:09:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:48.691075 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a90cd-393b-4936-90f6-e450f53cb47d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:48.716390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d5ba1-30c9-4fe9-ae70-5a5ec71852d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:48.717116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d5ba1-30c9-4fe9-ae70-5a5ec71852d5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:48.717395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d5ba1-30c9-4fe9-ae70-5a5ec71852d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:48.717668 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d5ba1-30c9-4fe9-ae70-5a5ec71852d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:48.723617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:48.728403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d5ba1-30c9-4fe9-ae70-5a5ec71852d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:48.729838 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d5ba1-30c9-4fe9-ae70-5a5ec71852d5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:48.744773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-851d5ba1-30c9-4fe9-ae70-5a5ec71852d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:48.744773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:48.744773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:48.744773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:48.744773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:48.744773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:48.744773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:48.744773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:48.744773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:48.744773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:48.744773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:48.744773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:48.744773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:48.744773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:48.744773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:48.744773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:48.745790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:48.746738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:48.746738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:48.746738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:48.746738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 549/1079] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:48.751759 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d5ba1-30c9-4fe9-ae70-5a5ec71852d5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:48.801737 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a90cd-393b-4936-90f6-e450f53cb47d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:48.836896 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113f8ed-0fa6-4bcc-9486-b68398264ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:48.837807 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113f8ed-0fa6-4bcc-9486-b68398264ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:48.838113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113f8ed-0fa6-4bcc-9486-b68398264ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:48.838502 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113f8ed-0fa6-4bcc-9486-b68398264ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:48.839164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113f8ed-0fa6-4bcc-9486-b68398264ea9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:48.839474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113f8ed-0fa6-4bcc-9486-b68398264ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:48.840978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113f8ed-0fa6-4bcc-9486-b68398264ea9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:48.850381 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 531/1080] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:09:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:48.855792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113f8ed-0fa6-4bcc-9486-b68398264ea9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:48.861656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d5ba1-30c9-4fe9-ae70-5a5ec71852d5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:48.886436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:48.897053 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94a121-996b-4f98-ae00-4956183ac6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:48.897641 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94a121-996b-4f98-ae00-4956183ac6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:48.897923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94a121-996b-4f98-ae00-4956183ac6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:48.898195 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94a121-996b-4f98-ae00-4956183ac6a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:48.898831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94a121-996b-4f98-ae00-4956183ac6a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:48.899133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94a121-996b-4f98-ae00-4956183ac6a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:48.900532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94a121-996b-4f98-ae00-4956183ac6a2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:48.913719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 550/1081] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:48.968135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:48.977691 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113f8ed-0fa6-4bcc-9486-b68398264ea9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:49.014800 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ec9f1-ddb2-4728-bca3-76bb812900a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:49.015454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ec9f1-ddb2-4728-bca3-76bb812900a9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:49.015762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ec9f1-ddb2-4728-bca3-76bb812900a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:49.016037 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ec9f1-ddb2-4728-bca3-76bb812900a9 None admin] PATH_INFO: `/v3/users/a1d4ea31b85b484c9dba0e773de9797b` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:49.017227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ec9f1-ddb2-4728-bca3-76bb812900a9 None admin] RBAC: Authorizing `identity:get_user(user_id=a1d4ea31b85b484c9dba0e773de9797b)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:49.018677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ec9f1-ddb2-4728-bca3-76bb812900a9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:49.021096 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 532/1082] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:09:48 2022] GET /identity/v3/users/a1d4ea31b85b484c9dba0e773de9797b => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:49.026833 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94a121-996b-4f98-ae00-4956183ac6a2 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:49.136621 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94a121-996b-4f98-ae00-4956183ac6a2 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:49.171621 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc682353-d14d-4a63-a401-34eb6ebdbdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:49.172161 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc682353-d14d-4a63-a401-34eb6ebdbdba None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:49.172438 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc682353-d14d-4a63-a401-34eb6ebdbdba None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:49.172711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc682353-d14d-4a63-a401-34eb6ebdbdba None admin] PATH_INFO: `/v3/projects/1df28446404b46ee8060b28c7c3da4be` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:49.173820 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc682353-d14d-4a63-a401-34eb6ebdbdba None admin] RBAC: Authorizing `identity:get_project(project_id=1df28446404b46ee8060b28c7c3da4be)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:49.175268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc682353-d14d-4a63-a401-34eb6ebdbdba None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:49.177158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 551/1083] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:09:49 2022] GET /identity/v3/projects/1df28446404b46ee8060b28c7c3da4be => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:49.183020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ec9f1-ddb2-4728-bca3-76bb812900a9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:49.291709 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ec9f1-ddb2-4728-bca3-76bb812900a9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:49.324684 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8c70a-17ca-4110-9364-3f9af878d539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:49.325338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8c70a-17ca-4110-9364-3f9af878d539 None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:49.325615 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8c70a-17ca-4110-9364-3f9af878d539 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:49.325884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8c70a-17ca-4110-9364-3f9af878d539 None admin] PATH_INFO: `/v3/projects/1df28446404b46ee8060b28c7c3da4be/users/a1d4ea31b85b484c9dba0e773de9797b/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:49.327789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba8c70a-17ca-4110-9364-3f9af878d539 None admin] RBAC: Authorizing `identity:create_grant(project_id=1df28446404b46ee8060b28c7c3da4be, user_id=a1d4ea31b85b484c9dba0e773de9797b, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:49.329204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba8c70a-17ca-4110-9364-3f9af878d539 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:49.342484 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 533/1084] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:09:49 2022] PUT /identity/v3/projects/1df28446404b46ee8060b28c7c3da4be/users/a1d4ea31b85b484c9dba0e773de9797b/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:09:49.600150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d2bbb-adfc-442d-a13e-a43db0b56d3d None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:49.600343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d2bbb-adfc-442d-a13e-a43db0b56d3d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:49.600604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d2bbb-adfc-442d-a13e-a43db0b56d3d None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:49.601960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 552/1085] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:49.607269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb98f6-bcf6-4ac1-864a-223d3c61e3fd None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:49.607745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb98f6-bcf6-4ac1-864a-223d3c61e3fd None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:49.607875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb98f6-bcf6-4ac1-864a-223d3c61e3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:49.625659 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c7bb98f6-bcf6-4ac1-864a-223d3c61e3fd None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:49.626843 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c7bb98f6-bcf6-4ac1-864a-223d3c61e3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:49.643762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:49.694319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 534/1086] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:49.961864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d2bbb-adfc-442d-a13e-a43db0b56d3d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:50.019010 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:50.065186 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d2bbb-adfc-442d-a13e-a43db0b56d3d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:50.098093 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523cc73e-8640-45f1-83fc-e9a282ccd811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:50.098712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cc73e-8640-45f1-83fc-e9a282ccd811 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:50.098990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cc73e-8640-45f1-83fc-e9a282ccd811 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:50.099260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cc73e-8640-45f1-83fc-e9a282ccd811 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:50.109158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523cc73e-8640-45f1-83fc-e9a282ccd811 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:50.114569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-523cc73e-8640-45f1-83fc-e9a282ccd811 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 12:09:50.114569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:50.114569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:50.114569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:50.114569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:50.114569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:50.114569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:50.114569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:50.114569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:50.114569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:50.114569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:50.114569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:50.114569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:50.114569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:50.114569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:50.114569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:50.115650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:50.115650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:50.115650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:50.115650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:50.115650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:50.115650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:50.115650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:50.115650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:50.115650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:50.115650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:50.115650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:50.115650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:50.115650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:50.115650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:50.115650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:50.115650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:50.116875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:50.116875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:50.116875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:50.116875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:50.116875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:50.116875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:50.116875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:50.116875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:50.116875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:50.116875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:50.116875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 12:09:50.116875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:50.116875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 553/1087] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Oct 6 12:09:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:09:50.121568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb98f6-bcf6-4ac1-864a-223d3c61e3fd None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:50.221997 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb98f6-bcf6-4ac1-864a-223d3c61e3fd None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:50.259576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2697115-9a1c-4d8a-af92-52a5b2a3a080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:50.260190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2697115-9a1c-4d8a-af92-52a5b2a3a080 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:50.260472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2697115-9a1c-4d8a-af92-52a5b2a3a080 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:50.260744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2697115-9a1c-4d8a-af92-52a5b2a3a080 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:50.261407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2697115-9a1c-4d8a-af92-52a5b2a3a080 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:50.261713 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2697115-9a1c-4d8a-af92-52a5b2a3a080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:50.263081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2697115-9a1c-4d8a-af92-52a5b2a3a080 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:50.271468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 535/1088] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Oct 6 12:09:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:50.277270 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523cc73e-8640-45f1-83fc-e9a282ccd811 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:50.374001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523cc73e-8640-45f1-83fc-e9a282ccd811 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:50.387990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:50.409481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cca65-bb09-472c-bcec-6114b6a85121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:50.410154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cca65-bb09-472c-bcec-6114b6a85121 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:50.410436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cca65-bb09-472c-bcec-6114b6a85121 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:50.410703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cca65-bb09-472c-bcec-6114b6a85121 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:50.421217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913cca65-bb09-472c-bcec-6114b6a85121 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:50.422653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913cca65-bb09-472c-bcec-6114b6a85121 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:50.436990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-913cca65-bb09-472c-bcec-6114b6a85121 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:50.436990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:50.436990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:50.436990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:50.436990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:50.436990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:50.436990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:50.436990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:50.436990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:50.436990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:50.436990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:50.436990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:50.436990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:50.436990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:50.436990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:50.436990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:50.438003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:50.439527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:50.439527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:50.439527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:50.439527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 554/1089] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:50.443893 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2697115-9a1c-4d8a-af92-52a5b2a3a080 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:50.460334 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:50.542971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2697115-9a1c-4d8a-af92-52a5b2a3a080 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:50.576443 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d011216-9b35-441e-9861-ebcd25639ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:50.577052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d011216-9b35-441e-9861-ebcd25639ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:50.577333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d011216-9b35-441e-9861-ebcd25639ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:50.577603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d011216-9b35-441e-9861-ebcd25639ee5 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:50.578232 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d011216-9b35-441e-9861-ebcd25639ee5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:50.578533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d011216-9b35-441e-9861-ebcd25639ee5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:50.579935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d011216-9b35-441e-9861-ebcd25639ee5 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:50.598356 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 536/1090] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:50.598539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cca65-bb09-472c-bcec-6114b6a85121 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:50.706103 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cca65-bb09-472c-bcec-6114b6a85121 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:50.742413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15494abf-f576-41bf-9b64-b72179f2e100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:50.743104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15494abf-f576-41bf-9b64-b72179f2e100 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:50.743473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15494abf-f576-41bf-9b64-b72179f2e100 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:50.743777 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15494abf-f576-41bf-9b64-b72179f2e100 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:50.754984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15494abf-f576-41bf-9b64-b72179f2e100 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:50.756450 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15494abf-f576-41bf-9b64-b72179f2e100 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:50.767806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-15494abf-f576-41bf-9b64-b72179f2e100 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:09:50.767806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:50.767806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:50.767806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:50.767806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:50.767806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:50.767806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:50.767806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:50.767806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:50.767806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:50.767806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:50.767806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:50.767806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:09:50.767806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:09:50.767806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:09:50.767806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:50.768899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:50.770562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:50.770562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:50.770562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:09:50.770562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:09:50.770562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:09:50.770562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:09:50.770562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:09:50.770562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:09:50.770562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 12:09:50.770562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:50.770562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 555/1091] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Oct 6 12:09:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:50.775797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d011216-9b35-441e-9861-ebcd25639ee5 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:50.848632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:50.874612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d011216-9b35-441e-9861-ebcd25639ee5 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:50.907581 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1437f5-9bdb-42a8-b032-3b1354812a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:50.908183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1437f5-9bdb-42a8-b032-3b1354812a7f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:50.908457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1437f5-9bdb-42a8-b032-3b1354812a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:50.908724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1437f5-9bdb-42a8-b032-3b1354812a7f None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:50.909435 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1437f5-9bdb-42a8-b032-3b1354812a7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:50.909734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1437f5-9bdb-42a8-b032-3b1354812a7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:50.911170 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1437f5-9bdb-42a8-b032-3b1354812a7f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:50.928148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 537/1092] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Oct 6 12:09:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:50.933942 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15494abf-f576-41bf-9b64-b72179f2e100 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:50.962210 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:51.033197 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15494abf-f576-41bf-9b64-b72179f2e100 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:51.066905 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9594e4-0a8e-4412-a92a-7c4b3d4be76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:51.067539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9594e4-0a8e-4412-a92a-7c4b3d4be76e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:51.067954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9594e4-0a8e-4412-a92a-7c4b3d4be76e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:51.068229 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9594e4-0a8e-4412-a92a-7c4b3d4be76e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:51.134119 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9594e4-0a8e-4412-a92a-7c4b3d4be76e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:51.135592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9594e4-0a8e-4412-a92a-7c4b3d4be76e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:51.160881 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:51.233065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 556/1093] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:09:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:09:51.239432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1437f5-9bdb-42a8-b032-3b1354812a7f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:51.337937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1437f5-9bdb-42a8-b032-3b1354812a7f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:51.367172 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:51.373289 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28f68d-065b-408c-8a9a-a10fc181de89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:51.373953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28f68d-065b-408c-8a9a-a10fc181de89 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:51.374233 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28f68d-065b-408c-8a9a-a10fc181de89 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:51.374503 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28f68d-065b-408c-8a9a-a10fc181de89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:51.385963 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28f68d-065b-408c-8a9a-a10fc181de89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:51.387355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28f68d-065b-408c-8a9a-a10fc181de89 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:51.404006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-af28f68d-065b-408c-8a9a-a10fc181de89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:51.404006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:51.404006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:51.404006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:51.404006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:51.404006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:51.404006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:51.404006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:51.404006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:51.404006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:51.404006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:51.404006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:51.404006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:09:51.404006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:09:51.404006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:09:51.404006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:51.405235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:09:51.406192 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:09:51.406192 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:09:51.406192 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:09:51.406192 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 538/1094] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:51.411609 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9594e4-0a8e-4412-a92a-7c4b3d4be76e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:51.496460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885abcb0-eb91-41b0-8c29-c449ddd99db8 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:51.496740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885abcb0-eb91-41b0-8c29-c449ddd99db8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:51.497014 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885abcb0-eb91-41b0-8c29-c449ddd99db8 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:51.498327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 539/1095] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:09:51.503138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb394f1-40a9-48bd-ac63-4f64febfa315 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:51.503417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb394f1-40a9-48bd-ac63-4f64febfa315 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:51.503741 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb394f1-40a9-48bd-ac63-4f64febfa315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:51.508849 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9594e4-0a8e-4412-a92a-7c4b3d4be76e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:51.521442 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-1bb394f1-40a9-48bd-ac63-4f64febfa315 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:51.522620 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1bb394f1-40a9-48bd-ac63-4f64febfa315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:51.536859 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:51.545904 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8962a5b-f0f4-4164-8382-6bd0c7e40939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:51.546504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8962a5b-f0f4-4164-8382-6bd0c7e40939 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:51.546779 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8962a5b-f0f4-4164-8382-6bd0c7e40939 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:51.547054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8962a5b-f0f4-4164-8382-6bd0c7e40939 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:51.547708 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8962a5b-f0f4-4164-8382-6bd0c7e40939 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:51.548009 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8962a5b-f0f4-4164-8382-6bd0c7e40939 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:51.549359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8962a5b-f0f4-4164-8382-6bd0c7e40939 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:51.561422 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 557/1096] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:51.567005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8962a5b-f0f4-4164-8382-6bd0c7e40939 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:51.589512 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 540/1097] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 12:09:51.662466 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8962a5b-f0f4-4164-8382-6bd0c7e40939 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:51.694208 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b63dee-abc5-4ca7-b570-fbe51b6770dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:51.694824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b63dee-abc5-4ca7-b570-fbe51b6770dc None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:51.695103 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b63dee-abc5-4ca7-b570-fbe51b6770dc None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:51.695375 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b63dee-abc5-4ca7-b570-fbe51b6770dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:51.701872 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b63dee-abc5-4ca7-b570-fbe51b6770dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:51.703299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b63dee-abc5-4ca7-b570-fbe51b6770dc None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:51.714799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-b8b63dee-abc5-4ca7-b570-fbe51b6770dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:51.714799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:51.714799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:51.714799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:51.714799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:51.714799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:51.714799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:51.714799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:51.714799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:51.714799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:51.714799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:51.714799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:51.714799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:09:51.714799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:09:51.714799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:51.716059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:51.717000 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:09:51.717000 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:09:51.717000 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:09:51.717000 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:09:51.717000 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:09:51.717000 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:09:51.717000 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:09:51.717000 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:51.717000 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 558/1098] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:09:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:51.721637 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb394f1-40a9-48bd-ac63-4f64febfa315 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:51.818144 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb394f1-40a9-48bd-ac63-4f64febfa315 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:51.831418 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:51.851212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52991b-cf85-4dae-b7af-34c5c83c86b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:51.851846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52991b-cf85-4dae-b7af-34c5c83c86b2 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:51.852127 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52991b-cf85-4dae-b7af-34c5c83c86b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:51.852399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52991b-cf85-4dae-b7af-34c5c83c86b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:51.853077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52991b-cf85-4dae-b7af-34c5c83c86b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:51.853380 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52991b-cf85-4dae-b7af-34c5c83c86b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:51.854804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52991b-cf85-4dae-b7af-34c5c83c86b2 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:51.861450 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b63dee-abc5-4ca7-b570-fbe51b6770dc None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:51.870835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 541/1099] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:09:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:51.872582 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52991b-cf85-4dae-b7af-34c5c83c86b2 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:51.908992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:51.963253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b63dee-abc5-4ca7-b570-fbe51b6770dc None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:51.970495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52991b-cf85-4dae-b7af-34c5c83c86b2 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:51.996667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ec988-d542-4a78-839a-d2ceb62d8c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:51.997286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ec988-d542-4a78-839a-d2ceb62d8c1c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:51.997559 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ec988-d542-4a78-839a-d2ceb62d8c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:51.997830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ec988-d542-4a78-839a-d2ceb62d8c1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:52.004943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decee6eb-7f26-413b-ae69-705b560ae52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:52.005545 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decee6eb-7f26-413b-ae69-705b560ae52a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:52.005825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decee6eb-7f26-413b-ae69-705b560ae52a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:52.006098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decee6eb-7f26-413b-ae69-705b560ae52a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:52.006811 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decee6eb-7f26-413b-ae69-705b560ae52a None admin] RBAC: Adding query filter params (role.id=14d78f802e8d43d684535749f98404c7, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=5a46e63d34564d34b6b5752706375798) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:52.007118 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decee6eb-7f26-413b-ae69-705b560ae52a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:52.008033 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ec988-d542-4a78-839a-d2ceb62d8c1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:09:52.008582 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decee6eb-7f26-413b-ae69-705b560ae52a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:52.013268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-1f0ec988-d542-4a78-839a-d2ceb62d8c1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:09:52.013268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:09:52.013268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:09:52.013268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:09:52.013268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:09:52.013268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:09:52.013268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:09:52.013268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:09:52.013268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:09:52.013268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:09:52.013268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:09:52.013268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:09:52.013268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:09:52.013268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:09:52.013268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:09:52.013268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:09:52.015646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:09:52.015646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:09:52.015646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:09:52.015646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:09:52.015646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:09:52.015646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:09:52.015646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:09:52.015646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:09:52.015646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:09:52.015646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:09:52.015646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:09:52.015646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:09:52.015646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:09:52.015646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:09:52.015646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:09:52.015646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:09:52.016595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:09:52.016595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:09:52.016595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:09:52.016595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:09:52.016595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:09:52.016595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:09:52.016595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:09:52.016595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:09:52.016595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:09:52.016595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:09:52.016595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:09:52.016595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:09:52.017250 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 542/1100] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:09:51 2022] GET /identity/v3/role_assignments?user.id=5a46e63d34564d34b6b5752706375798&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=14d78f802e8d43d684535749f98404c7 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:09:52.017325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 559/1101] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:09:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:09:52.022481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decee6eb-7f26-413b-ae69-705b560ae52a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:52.064670 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:52.127895 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decee6eb-7f26-413b-ae69-705b560ae52a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:52.166088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74cee9-1007-419b-9e11-a279a5275206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:52.166868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74cee9-1007-419b-9e11-a279a5275206 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:52.167233 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74cee9-1007-419b-9e11-a279a5275206 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:52.167614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74cee9-1007-419b-9e11-a279a5275206 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:52.168293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d74cee9-1007-419b-9e11-a279a5275206 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:52.168599 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d74cee9-1007-419b-9e11-a279a5275206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:52.169996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d74cee9-1007-419b-9e11-a279a5275206 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:52.178855 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 543/1102] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:09:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:52.185002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ec988-d542-4a78-839a-d2ceb62d8c1c None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:52.296443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ec988-d542-4a78-839a-d2ceb62d8c1c None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:52.303341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:52.330959 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f672eda-2cda-4ead-a478-8e3e15fffa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:52.331623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f672eda-2cda-4ead-a478-8e3e15fffa69 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:52.332434 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f672eda-2cda-4ead-a478-8e3e15fffa69 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:52.332710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f672eda-2cda-4ead-a478-8e3e15fffa69 None admin] PATH_INFO: `/v3/users/a1d4ea31b85b484c9dba0e773de9797b` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:52.333847 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f672eda-2cda-4ead-a478-8e3e15fffa69 None admin] RBAC: Authorizing `identity:get_user(user_id=a1d4ea31b85b484c9dba0e773de9797b)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:52.335276 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f672eda-2cda-4ead-a478-8e3e15fffa69 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:52.337097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 560/1103] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:09:52 2022] GET /identity/v3/users/a1d4ea31b85b484c9dba0e773de9797b => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:52.342666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74cee9-1007-419b-9e11-a279a5275206 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:52.444549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:52.447739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74cee9-1007-419b-9e11-a279a5275206 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:52.483172 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738d77a-aed5-4f87-a21d-cd3ab348bfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:52.483919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738d77a-aed5-4f87-a21d-cd3ab348bfd8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:52.484209 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738d77a-aed5-4f87-a21d-cd3ab348bfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:52.484482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738d77a-aed5-4f87-a21d-cd3ab348bfd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:52.552637 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738d77a-aed5-4f87-a21d-cd3ab348bfd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:52.554028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738d77a-aed5-4f87-a21d-cd3ab348bfd8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:52.649127 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:52.652842 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 544/1104] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:09:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:09:52.658612 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f672eda-2cda-4ead-a478-8e3e15fffa69 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:52.766215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f672eda-2cda-4ead-a478-8e3e15fffa69 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:52.800360 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6b676-2552-4ef1-9b6f-ec9c3e2443b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:52.801001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6b676-2552-4ef1-9b6f-ec9c3e2443b8 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:52.801285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6b676-2552-4ef1-9b6f-ec9c3e2443b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:52.801560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6b676-2552-4ef1-9b6f-ec9c3e2443b8 None admin] PATH_INFO: `/v3/projects/1df28446404b46ee8060b28c7c3da4be` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:52.802731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6b676-2552-4ef1-9b6f-ec9c3e2443b8 None admin] RBAC: Authorizing `identity:get_project(project_id=1df28446404b46ee8060b28c7c3da4be)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:52.804152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6b676-2552-4ef1-9b6f-ec9c3e2443b8 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:52.805796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 561/1105] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:09:52 2022] GET /identity/v3/projects/1df28446404b46ee8060b28c7c3da4be => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:52.812085 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738d77a-aed5-4f87-a21d-cd3ab348bfd8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:52.913714 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738d77a-aed5-4f87-a21d-cd3ab348bfd8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:52.947094 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885af866-884a-48b2-8da3-23cdc6676254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:52.947868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885af866-884a-48b2-8da3-23cdc6676254 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:52.948276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885af866-884a-48b2-8da3-23cdc6676254 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:52.948685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885af866-884a-48b2-8da3-23cdc6676254 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:52.949400 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885af866-884a-48b2-8da3-23cdc6676254 None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=1df28446404b46ee8060b28c7c3da4be, user.id=a1d4ea31b85b484c9dba0e773de9797b) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:52.949707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885af866-884a-48b2-8da3-23cdc6676254 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:52.951123 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885af866-884a-48b2-8da3-23cdc6676254 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:52.957333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 545/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:09:52 2022] GET /identity/v3/role_assignments?user.id=a1d4ea31b85b484c9dba0e773de9797b&scope.project.id=1df28446404b46ee8060b28c7c3da4be&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:55.083145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f1e91-1c19-4818-a33f-a6c804b225cc None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:55.083713 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f1e91-1c19-4818-a33f-a6c804b225cc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:55.083713 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f1e91-1c19-4818-a33f-a6c804b225cc None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:55.085033 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 562/1107] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:55.090527 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb461df-f1e9-45f9-90f6-96c32e3bfaf8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:55.090938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb461df-f1e9-45f9-90f6-96c32e3bfaf8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:55.091107 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb461df-f1e9-45f9-90f6-96c32e3bfaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:55.112770 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7cb461df-f1e9-45f9-90f6-96c32e3bfaf8 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:55.114491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7cb461df-f1e9-45f9-90f6-96c32e3bfaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:55.169428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:55.178401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 546/1108] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:55.457129 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f1e91-1c19-4818-a33f-a6c804b225cc None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:55.500736 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:55.558204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f1e91-1c19-4818-a33f-a6c804b225cc None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:55.590139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937a339-4532-4645-bd63-19ccdda4e88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:55.590748 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937a339-4532-4645-bd63-19ccdda4e88c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:55.591026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937a339-4532-4645-bd63-19ccdda4e88c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:55.591292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937a339-4532-4645-bd63-19ccdda4e88c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:55.592409 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937a339-4532-4645-bd63-19ccdda4e88c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:55.593828 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937a339-4532-4645-bd63-19ccdda4e88c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:55.595363 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 563/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:09:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:55.601038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb461df-f1e9-45f9-90f6-96c32e3bfaf8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:55.695416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb461df-f1e9-45f9-90f6-96c32e3bfaf8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:55.728509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e588e6-148e-4f7a-8020-b791fdfae8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:55.729038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e588e6-148e-4f7a-8020-b791fdfae8df None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:55.729318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e588e6-148e-4f7a-8020-b791fdfae8df None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:55.729588 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e588e6-148e-4f7a-8020-b791fdfae8df None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:55.730223 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e588e6-148e-4f7a-8020-b791fdfae8df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:55.731665 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e588e6-148e-4f7a-8020-b791fdfae8df None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:55.752194 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 547/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:09:55 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:09:57.910804 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3419e-570c-4379-b23e-aae436b1520d None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:57.911358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3419e-570c-4379-b23e-aae436b1520d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:57.911358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3419e-570c-4379-b23e-aae436b1520d None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:57.912718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 564/1111] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:09:57.917951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f234c8-e3f6-4e8c-8152-668db03d4e2b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:57.918337 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f234c8-e3f6-4e8c-8152-668db03d4e2b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:57.918495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f234c8-e3f6-4e8c-8152-668db03d4e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:57.936113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-95f234c8-e3f6-4e8c-8152-668db03d4e2b None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:09:57.937279 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-95f234c8-e3f6-4e8c-8152-668db03d4e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:09:57.982097 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:58.002742 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 548/1112] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:09:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:09:58.274150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3419e-570c-4379-b23e-aae436b1520d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:58.372598 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3419e-570c-4379-b23e-aae436b1520d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:58.382895 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:58.405747 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff7e25-e7f0-4c91-af78-6c4326d45a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:58.406364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7e25-e7f0-4c91-af78-6c4326d45a0d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:58.406646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7e25-e7f0-4c91-af78-6c4326d45a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:58.406923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7e25-e7f0-4c91-af78-6c4326d45a0d None admin] PATH_INFO: `/v3/roles/2c11a95c1ecf4988a95aad8d96ea56e9` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:58.408330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff7e25-e7f0-4c91-af78-6c4326d45a0d None admin] RBAC: Authorizing `identity:get_role(role_id=2c11a95c1ecf4988a95aad8d96ea56e9)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:58.409717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff7e25-e7f0-4c91-af78-6c4326d45a0d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:58.411025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 565/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:09:58 2022] GET /identity/v3/roles/2c11a95c1ecf4988a95aad8d96ea56e9 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:58.416739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f234c8-e3f6-4e8c-8152-668db03d4e2b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:58.511382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f234c8-e3f6-4e8c-8152-668db03d4e2b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:58.544443 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356108eb-8a72-4049-92ee-215f160dbcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:58.545058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356108eb-8a72-4049-92ee-215f160dbcad None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:58.545339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356108eb-8a72-4049-92ee-215f160dbcad None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:58.545612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356108eb-8a72-4049-92ee-215f160dbcad None admin] PATH_INFO: `/v3/users/9d14519603504cd690931b11506a6cad` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:58.558058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356108eb-8a72-4049-92ee-215f160dbcad None admin] RBAC: Authorizing `identity:get_user(user_id=9d14519603504cd690931b11506a6cad)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:58.559496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356108eb-8a72-4049-92ee-215f160dbcad None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:58.561083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 549/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:09:58 2022] GET /identity/v3/users/9d14519603504cd690931b11506a6cad => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:58.566732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff7e25-e7f0-4c91-af78-6c4326d45a0d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:58.664187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff7e25-e7f0-4c91-af78-6c4326d45a0d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:58.696232 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df855e82-56ad-4eba-8151-efb3d90f3482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:58.696820 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df855e82-56ad-4eba-8151-efb3d90f3482 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:58.697104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df855e82-56ad-4eba-8151-efb3d90f3482 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:58.697382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df855e82-56ad-4eba-8151-efb3d90f3482 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:58.742366 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:58.764029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df855e82-56ad-4eba-8151-efb3d90f3482 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:58.765412 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df855e82-56ad-4eba-8151-efb3d90f3482 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:58.863180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 566/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:09:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:09:58.869012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356108eb-8a72-4049-92ee-215f160dbcad None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:58.907903 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:58.965372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356108eb-8a72-4049-92ee-215f160dbcad None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:58.997179 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de1787-4855-4dce-93c0-3c8710103a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:58.997791 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de1787-4855-4dce-93c0-3c8710103a4e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:58.998072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de1787-4855-4dce-93c0-3c8710103a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:58.998344 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de1787-4855-4dce-93c0-3c8710103a4e None admin] PATH_INFO: `/v3/projects/1df28446404b46ee8060b28c7c3da4be` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:58.999444 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de1787-4855-4dce-93c0-3c8710103a4e None admin] RBAC: Authorizing `identity:get_project(project_id=1df28446404b46ee8060b28c7c3da4be)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:59.000883 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de1787-4855-4dce-93c0-3c8710103a4e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:59.002463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 550/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:09:58 2022] GET /identity/v3/projects/1df28446404b46ee8060b28c7c3da4be => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:09:59.009113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df855e82-56ad-4eba-8151-efb3d90f3482 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:09:59.085365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:09:59.106414 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df855e82-56ad-4eba-8151-efb3d90f3482 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:09:59.147892 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a256c-3548-44b9-b841-d776cd3b8fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:09:59.148485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a256c-3548-44b9-b841-d776cd3b8fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:09:59.148770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a256c-3548-44b9-b841-d776cd3b8fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:09:59.149050 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a256c-3548-44b9-b841-d776cd3b8fb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:09:59.149795 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a256c-3548-44b9-b841-d776cd3b8fb5 None admin] RBAC: Adding query filter params (role.id=2c11a95c1ecf4988a95aad8d96ea56e9, scope.project.id=1df28446404b46ee8060b28c7c3da4be, user.id=9d14519603504cd690931b11506a6cad) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:09:59.150105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a256c-3548-44b9-b841-d776cd3b8fb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:09:59.151544 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a256c-3548-44b9-b841-d776cd3b8fb5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:09:59.156964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 567/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:09:59 2022] GET /identity/v3/role_assignments?user.id=9d14519603504cd690931b11506a6cad&scope.project.id=1df28446404b46ee8060b28c7c3da4be&role.id=2c11a95c1ecf4988a95aad8d96ea56e9 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:01.233203 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512260f-8f8b-4cd7-99aa-5401e54a368c None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:01.233760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512260f-8f8b-4cd7-99aa-5401e54a368c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:01.233760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512260f-8f8b-4cd7-99aa-5401e54a368c None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:01.235061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 551/1118] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:10:01.240247 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9d435-b6ec-4f87-8747-d869188bb3ab None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:01.240693 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9d435-b6ec-4f87-8747-d869188bb3ab None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:01.240788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9d435-b6ec-4f87-8747-d869188bb3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:01.259288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-43c9d435-b6ec-4f87-8747-d869188bb3ab None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:10:01.260489 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-43c9d435-b6ec-4f87-8747-d869188bb3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:10:01.323228 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 568/1119] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:10:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:10:01.588220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512260f-8f8b-4cd7-99aa-5401e54a368c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:01.684607 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512260f-8f8b-4cd7-99aa-5401e54a368c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:01.691590 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:01.717276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414eae8-d9fc-4b2c-ac5a-03f64be87d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:01.717890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414eae8-d9fc-4b2c-ac5a-03f64be87d03 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:01.718166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414eae8-d9fc-4b2c-ac5a-03f64be87d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:01.718434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414eae8-d9fc-4b2c-ac5a-03f64be87d03 None admin] PATH_INFO: `/v3/roles/2c11a95c1ecf4988a95aad8d96ea56e9` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:01.719804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414eae8-d9fc-4b2c-ac5a-03f64be87d03 None admin] RBAC: Authorizing `identity:get_role(role_id=2c11a95c1ecf4988a95aad8d96ea56e9)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:01.721153 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414eae8-d9fc-4b2c-ac5a-03f64be87d03 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:01.722979 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 552/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:10:01 2022] GET /identity/v3/roles/2c11a95c1ecf4988a95aad8d96ea56e9 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:01.728768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9d435-b6ec-4f87-8747-d869188bb3ab None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:01.796248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:01.824813 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9d435-b6ec-4f87-8747-d869188bb3ab None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:01.857856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb301737-724e-4e8b-86bb-6e06951e9b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:01.858474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb301737-724e-4e8b-86bb-6e06951e9b5a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:01.858760 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb301737-724e-4e8b-86bb-6e06951e9b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:01.859032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb301737-724e-4e8b-86bb-6e06951e9b5a None admin] PATH_INFO: `/v3/users/9d14519603504cd690931b11506a6cad` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:01.860163 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb301737-724e-4e8b-86bb-6e06951e9b5a None admin] RBAC: Authorizing `identity:get_user(user_id=9d14519603504cd690931b11506a6cad)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:01.861578 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb301737-724e-4e8b-86bb-6e06951e9b5a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:01.863167 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 569/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:10:01 2022] GET /identity/v3/users/9d14519603504cd690931b11506a6cad => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:01.868873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414eae8-d9fc-4b2c-ac5a-03f64be87d03 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:01.966446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414eae8-d9fc-4b2c-ac5a-03f64be87d03 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:01.998205 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7e382-53e8-4076-97a1-4983c0bd6761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:01.998815 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7e382-53e8-4076-97a1-4983c0bd6761 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:01.999097 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7e382-53e8-4076-97a1-4983c0bd6761 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:01.999370 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7e382-53e8-4076-97a1-4983c0bd6761 None admin] PATH_INFO: `/v3/projects/1df28446404b46ee8060b28c7c3da4be` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:02.000497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7e382-53e8-4076-97a1-4983c0bd6761 None admin] RBAC: Authorizing `identity:get_project(project_id=1df28446404b46ee8060b28c7c3da4be)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:02.001919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7e382-53e8-4076-97a1-4983c0bd6761 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:02.004056 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 553/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:10:01 2022] GET /identity/v3/projects/1df28446404b46ee8060b28c7c3da4be => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:02.009805 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb301737-724e-4e8b-86bb-6e06951e9b5a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:02.104925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb301737-724e-4e8b-86bb-6e06951e9b5a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:02.137410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdfeae-56f1-45d2-b498-737ca3061bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:02.137985 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdfeae-56f1-45d2-b498-737ca3061bf4 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:02.138263 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdfeae-56f1-45d2-b498-737ca3061bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:02.138535 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdfeae-56f1-45d2-b498-737ca3061bf4 None admin] PATH_INFO: `/v3/projects/1df28446404b46ee8060b28c7c3da4be/users/9d14519603504cd690931b11506a6cad/roles/2c11a95c1ecf4988a95aad8d96ea56e9` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:02.140617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdfeae-56f1-45d2-b498-737ca3061bf4 None admin] RBAC: Authorizing `identity:create_grant(project_id=1df28446404b46ee8060b28c7c3da4be, user_id=9d14519603504cd690931b11506a6cad, role_id=2c11a95c1ecf4988a95aad8d96ea56e9)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:02.142028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdfeae-56f1-45d2-b498-737ca3061bf4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:02.150607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:02.155858 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 570/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:10:02 2022] PUT /identity/v3/projects/1df28446404b46ee8060b28c7c3da4be/users/9d14519603504cd690931b11506a6cad/roles/2c11a95c1ecf4988a95aad8d96ea56e9 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:10:04.236702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f866d21-308e-46e0-ae98-04ed87d0ac28 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:04.237786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f866d21-308e-46e0-ae98-04ed87d0ac28 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:04.237900 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f866d21-308e-46e0-ae98-04ed87d0ac28 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:04.239175 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 554/1124] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:10:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:10:04.244539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e14a34-4f87-4aac-a99d-6d89ecef972b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:04.244930 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e14a34-4f87-4aac-a99d-6d89ecef972b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:04.245084 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e14a34-4f87-4aac-a99d-6d89ecef972b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:04.262871 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-15e14a34-4f87-4aac-a99d-6d89ecef972b None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:10:04.264072 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-15e14a34-4f87-4aac-a99d-6d89ecef972b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:10:04.329817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 571/1125] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:10:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:10:04.598396 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f866d21-308e-46e0-ae98-04ed87d0ac28 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:04.641494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:04.697682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f866d21-308e-46e0-ae98-04ed87d0ac28 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:04.730709 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3391716-97aa-45af-8180-a04bba13608f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:04.731316 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3391716-97aa-45af-8180-a04bba13608f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:04.731624 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3391716-97aa-45af-8180-a04bba13608f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:04.731898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3391716-97aa-45af-8180-a04bba13608f None admin] PATH_INFO: `/v3/roles/2c11a95c1ecf4988a95aad8d96ea56e9` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:04.733268 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3391716-97aa-45af-8180-a04bba13608f None admin] RBAC: Authorizing `identity:get_role(role_id=2c11a95c1ecf4988a95aad8d96ea56e9)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:04.734625 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3391716-97aa-45af-8180-a04bba13608f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:04.735942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 555/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:10:04 2022] GET /identity/v3/roles/2c11a95c1ecf4988a95aad8d96ea56e9 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:04.741881 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e14a34-4f87-4aac-a99d-6d89ecef972b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:04.837244 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e14a34-4f87-4aac-a99d-6d89ecef972b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:04.864080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:04.869937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d962e1-3dcb-44b1-b54a-ec5d369537dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:04.870556 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d962e1-3dcb-44b1-b54a-ec5d369537dd None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:04.870838 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d962e1-3dcb-44b1-b54a-ec5d369537dd None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:04.871113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d962e1-3dcb-44b1-b54a-ec5d369537dd None admin] PATH_INFO: `/v3/users/9d14519603504cd690931b11506a6cad` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:04.872251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d962e1-3dcb-44b1-b54a-ec5d369537dd None admin] RBAC: Authorizing `identity:get_user(user_id=9d14519603504cd690931b11506a6cad)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:04.873675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d962e1-3dcb-44b1-b54a-ec5d369537dd None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:04.875260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 572/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:10:04 2022] GET /identity/v3/users/9d14519603504cd690931b11506a6cad => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:04.881264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3391716-97aa-45af-8180-a04bba13608f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:04.977449 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3391716-97aa-45af-8180-a04bba13608f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:04.987127 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:05.010490 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a1fc9-029e-4a67-a9b5-61a85aa148ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:05.011084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a1fc9-029e-4a67-a9b5-61a85aa148ec None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:05.011367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a1fc9-029e-4a67-a9b5-61a85aa148ec None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:05.011673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a1fc9-029e-4a67-a9b5-61a85aa148ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:05.075821 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a1fc9-029e-4a67-a9b5-61a85aa148ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:05.077252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a1fc9-029e-4a67-a9b5-61a85aa148ec None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:05.170490 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 556/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:10:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:10:05.176773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d962e1-3dcb-44b1-b54a-ec5d369537dd None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:05.271988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d962e1-3dcb-44b1-b54a-ec5d369537dd None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:05.304959 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4403e-ac5e-4063-bc75-c89fa308f47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:05.305569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4403e-ac5e-4063-bc75-c89fa308f47f None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:05.305845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4403e-ac5e-4063-bc75-c89fa308f47f None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:05.306113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4403e-ac5e-4063-bc75-c89fa308f47f None admin] PATH_INFO: `/v3/projects/1df28446404b46ee8060b28c7c3da4be` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:05.307207 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4403e-ac5e-4063-bc75-c89fa308f47f None admin] RBAC: Authorizing `identity:get_project(project_id=1df28446404b46ee8060b28c7c3da4be)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:05.308656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4403e-ac5e-4063-bc75-c89fa308f47f None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:05.310258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 573/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:10:05 2022] GET /identity/v3/projects/1df28446404b46ee8060b28c7c3da4be => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:05.316849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a1fc9-029e-4a67-a9b5-61a85aa148ec None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:05.328005 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:05.411973 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a1fc9-029e-4a67-a9b5-61a85aa148ec None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:05.445380 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d336b15-18c7-49cf-82f4-4df7fba09b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:05.445915 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d336b15-18c7-49cf-82f4-4df7fba09b70 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:05.446195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d336b15-18c7-49cf-82f4-4df7fba09b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:05.446467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d336b15-18c7-49cf-82f4-4df7fba09b70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:05.447172 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d336b15-18c7-49cf-82f4-4df7fba09b70 None admin] RBAC: Adding query filter params (role.id=2c11a95c1ecf4988a95aad8d96ea56e9, scope.project.id=1df28446404b46ee8060b28c7c3da4be, user.id=9d14519603504cd690931b11506a6cad) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:10:05.447478 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d336b15-18c7-49cf-82f4-4df7fba09b70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:05.448945 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d336b15-18c7-49cf-82f4-4df7fba09b70 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:05.454460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 557/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:10:05 2022] GET /identity/v3/role_assignments?user.id=9d14519603504cd690931b11506a6cad&scope.project.id=1df28446404b46ee8060b28c7c3da4be&role.id=2c11a95c1ecf4988a95aad8d96ea56e9 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:07.582695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746940f9-6a8f-4ae7-b32e-45ab82b9a76d None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:07.583207 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746940f9-6a8f-4ae7-b32e-45ab82b9a76d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:07.583207 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746940f9-6a8f-4ae7-b32e-45ab82b9a76d None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:07.584520 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 574/1131] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:10:07.589886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5ba44-131c-4bd0-bc60-b933f745cb7f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:07.590304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5ba44-131c-4bd0-bc60-b933f745cb7f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:07.590431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5ba44-131c-4bd0-bc60-b933f745cb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:07.608638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-3ca5ba44-131c-4bd0-bc60-b933f745cb7f None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:10:07.609802 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3ca5ba44-131c-4bd0-bc60-b933f745cb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:10:07.672659 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 558/1132] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:10:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:10:07.943518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746940f9-6a8f-4ae7-b32e-45ab82b9a76d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:08.008124 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:08.042361 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746940f9-6a8f-4ae7-b32e-45ab82b9a76d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:08.074400 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bbeff-2c68-4fe5-bc9f-dfb8f612ac51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:08.075008 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bbeff-2c68-4fe5-bc9f-dfb8f612ac51 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:08.075284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bbeff-2c68-4fe5-bc9f-dfb8f612ac51 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:08.075576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bbeff-2c68-4fe5-bc9f-dfb8f612ac51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:08.076662 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93bbeff-2c68-4fe5-bc9f-dfb8f612ac51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:08.078070 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93bbeff-2c68-4fe5-bc9f-dfb8f612ac51 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:08.079621 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 575/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:10:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:08.086166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5ba44-131c-4bd0-bc60-b933f745cb7f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:08.089280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:08.191223 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5ba44-131c-4bd0-bc60-b933f745cb7f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:08.223885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795e2e1-e6ab-4e87-8c52-0b1737544ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:08.224425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795e2e1-e6ab-4e87-8c52-0b1737544ec9 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:08.224708 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795e2e1-e6ab-4e87-8c52-0b1737544ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:08.224988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795e2e1-e6ab-4e87-8c52-0b1737544ec9 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:08.225631 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795e2e1-e6ab-4e87-8c52-0b1737544ec9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:08.227053 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795e2e1-e6ab-4e87-8c52-0b1737544ec9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:08.253548 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 559/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Oct 6 12:10:08 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:10:10.397695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c649-e71f-49b3-8afa-2ce409c15bc0 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:10.398217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c649-e71f-49b3-8afa-2ce409c15bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:10.398217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c649-e71f-49b3-8afa-2ce409c15bc0 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:10.399593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 576/1135] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:10:10.404900 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ce9e3-bdd9-4684-bc38-7eb0f0444fa4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:10.405327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ce9e3-bdd9-4684-bc38-7eb0f0444fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:10.405437 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ce9e3-bdd9-4684-bc38-7eb0f0444fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:10.423634 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-846ce9e3-bdd9-4684-bc38-7eb0f0444fa4 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:10:10.424795 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-846ce9e3-bdd9-4684-bc38-7eb0f0444fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:10:10.472761 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:10.489427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 560/1136] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:10:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:10:10.764024 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344c649-e71f-49b3-8afa-2ce409c15bc0 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:10.862790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344c649-e71f-49b3-8afa-2ce409c15bc0 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:10.894776 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fec20-4c0b-46fe-abd3-17c3e9f9333c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:10.895394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fec20-4c0b-46fe-abd3-17c3e9f9333c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:10.895707 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fec20-4c0b-46fe-abd3-17c3e9f9333c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:10.895979 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fec20-4c0b-46fe-abd3-17c3e9f9333c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:10.897080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62fec20-4c0b-46fe-abd3-17c3e9f9333c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:10.898497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62fec20-4c0b-46fe-abd3-17c3e9f9333c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:10.900051 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 577/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:10:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:10.905804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ce9e3-bdd9-4684-bc38-7eb0f0444fa4 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:11.002634 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ce9e3-bdd9-4684-bc38-7eb0f0444fa4 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:11.046715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b5785-ce89-4186-9274-994169bf00a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:11.047249 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b5785-ce89-4186-9274-994169bf00a1 None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:11.048747 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b5785-ce89-4186-9274-994169bf00a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:11.049057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b5785-ce89-4186-9274-994169bf00a1 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:11.049671 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b5785-ce89-4186-9274-994169bf00a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:11.051686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b5785-ce89-4186-9274-994169bf00a1 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:11.072448 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 561/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:10:10 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:10:13.221670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dd9f0-6387-4b6f-b9e3-c408b4b066ab None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:13.222200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dd9f0-6387-4b6f-b9e3-c408b4b066ab None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:13.222200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dd9f0-6387-4b6f-b9e3-c408b4b066ab None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:13.223569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 578/1139] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:10:13.228705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe7803-7987-4a73-9040-a3746c357c6e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:13.229112 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe7803-7987-4a73-9040-a3746c357c6e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:13.229252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe7803-7987-4a73-9040-a3746c357c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:13.247079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d7fe7803-7987-4a73-9040-a3746c357c6e None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:10:13.248314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d7fe7803-7987-4a73-9040-a3746c357c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:10:13.282924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:13.314126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 562/1140] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:10:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:10:13.586414 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5dd9f0-6387-4b6f-b9e3-c408b4b066ab None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:13.589238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:13.684913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5dd9f0-6387-4b6f-b9e3-c408b4b066ab None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:13.717275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04166709-0289-440c-9dc3-235099d3f726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:13.717874 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04166709-0289-440c-9dc3-235099d3f726 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:13.718151 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04166709-0289-440c-9dc3-235099d3f726 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:13.718420 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04166709-0289-440c-9dc3-235099d3f726 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:13.728604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04166709-0289-440c-9dc3-235099d3f726 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:10:13.733288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-04166709-0289-440c-9dc3-235099d3f726 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:10:13.733288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:10:13.733288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:10:13.733288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:10:13.733288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:10:13.733288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:10:13.733288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:10:13.733288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:10:13.733288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:10:13.733288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:10:13.733288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:10:13.733288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:10:13.733288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:10:13.733288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:10:13.733288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:10:13.733288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:10:13.734480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:10:13.734480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:10:13.734480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:10:13.734480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:10:13.734480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:10:13.734480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:10:13.734480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:10:13.734480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:10:13.734480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:10:13.734480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:10:13.734480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:10:13.734480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:10:13.734480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:10:13.734480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:10:13.734480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:10:13.734480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:10:13.735706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:10:13.735706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:10:13.735706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:10:13.735706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:10:13.735706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:10:13.735706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:10:13.735706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:10:13.735706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:10:13.735706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:10:13.735706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:10:13.735706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:10:13.735706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:10:13.735706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 579/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:10:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:10:13.740303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe7803-7987-4a73-9040-a3746c357c6e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:13.835715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe7803-7987-4a73-9040-a3746c357c6e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:13.867537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bc8c7-ce1d-4e38-963e-fea23d5e17db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:13.868132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bc8c7-ce1d-4e38-963e-fea23d5e17db None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:13.868414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bc8c7-ce1d-4e38-963e-fea23d5e17db None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:13.868687 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bc8c7-ce1d-4e38-963e-fea23d5e17db None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:13.869342 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05bc8c7-ce1d-4e38-963e-fea23d5e17db None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:10:13.869646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05bc8c7-ce1d-4e38-963e-fea23d5e17db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:13.871010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05bc8c7-ce1d-4e38-963e-fea23d5e17db None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:13.879435 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 563/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:10:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:13.885122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04166709-0289-440c-9dc3-235099d3f726 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:13.945591 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:13.983185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04166709-0289-440c-9dc3-235099d3f726 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:14.015576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c116ce-d7d3-42f0-9ba2-785d8198e2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:14.016186 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c116ce-d7d3-42f0-9ba2-785d8198e2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:14.016469 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c116ce-d7d3-42f0-9ba2-785d8198e2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:14.016736 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c116ce-d7d3-42f0-9ba2-785d8198e2e6 None admin] PATH_INFO: `/v3/users/7bf0717e809d403e9712ab1de6ea5a62` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:14.030047 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c116ce-d7d3-42f0-9ba2-785d8198e2e6 None admin] RBAC: Authorizing `identity:get_user(user_id=7bf0717e809d403e9712ab1de6ea5a62)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:14.031467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c116ce-d7d3-42f0-9ba2-785d8198e2e6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:14.033060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 580/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:10:13 2022] GET /identity/v3/users/7bf0717e809d403e9712ab1de6ea5a62 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:14.038756 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bc8c7-ce1d-4e38-963e-fea23d5e17db None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:14.072298 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:14.137669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bc8c7-ce1d-4e38-963e-fea23d5e17db None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:14.170120 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e4ccc-f97d-4802-addc-7bbf82fd07e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:14.170717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e4ccc-f97d-4802-addc-7bbf82fd07e5 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:14.171002 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e4ccc-f97d-4802-addc-7bbf82fd07e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:14.171276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e4ccc-f97d-4802-addc-7bbf82fd07e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:14.238389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e4ccc-f97d-4802-addc-7bbf82fd07e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:14.239872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e4ccc-f97d-4802-addc-7bbf82fd07e5 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:14.271783 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:14.335906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 564/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:10:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:10:14.341824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c116ce-d7d3-42f0-9ba2-785d8198e2e6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:14.438383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c116ce-d7d3-42f0-9ba2-785d8198e2e6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:14.459258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:14.473626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580f5d2-2349-44f7-9e31-5826b6f36aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:14.474296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580f5d2-2349-44f7-9e31-5826b6f36aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:14.474582 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580f5d2-2349-44f7-9e31-5826b6f36aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:14.474853 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580f5d2-2349-44f7-9e31-5826b6f36aa5 None admin] PATH_INFO: `/v3/projects/f03af4c2207945a785107a7e3123ee97` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:14.475983 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580f5d2-2349-44f7-9e31-5826b6f36aa5 None admin] RBAC: Authorizing `identity:get_project(project_id=f03af4c2207945a785107a7e3123ee97)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:14.477398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580f5d2-2349-44f7-9e31-5826b6f36aa5 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:14.479003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 581/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:10:14 2022] GET /identity/v3/projects/f03af4c2207945a785107a7e3123ee97 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:14.485168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e4ccc-f97d-4802-addc-7bbf82fd07e5 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:14.580438 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e4ccc-f97d-4802-addc-7bbf82fd07e5 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:14.614093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a07f73-b549-466c-9123-50ea1ce59c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:14.614685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a07f73-b549-466c-9123-50ea1ce59c6e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:14.614965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a07f73-b549-466c-9123-50ea1ce59c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:14.615238 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a07f73-b549-466c-9123-50ea1ce59c6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:14.615980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a07f73-b549-466c-9123-50ea1ce59c6e None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=f03af4c2207945a785107a7e3123ee97, user.id=7bf0717e809d403e9712ab1de6ea5a62) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:10:14.616287 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a07f73-b549-466c-9123-50ea1ce59c6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:14.617707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a07f73-b549-466c-9123-50ea1ce59c6e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:14.619386 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:14.624069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 565/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:10:14 2022] GET /identity/v3/role_assignments?user.id=7bf0717e809d403e9712ab1de6ea5a62&scope.project.id=f03af4c2207945a785107a7e3123ee97&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:16.704709 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcb052-3347-4e49-bafe-24d9806fd329 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:16.705734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcb052-3347-4e49-bafe-24d9806fd329 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:16.706060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcb052-3347-4e49-bafe-24d9806fd329 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:16.707402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 582/1147] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:10:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:10:16.712784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd48adf-6f0e-479b-a485-363d23a839e4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:16.713224 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd48adf-6f0e-479b-a485-363d23a839e4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:16.713318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd48adf-6f0e-479b-a485-363d23a839e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:16.731700 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-cbd48adf-6f0e-479b-a485-363d23a839e4 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:10:16.732868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-cbd48adf-6f0e-479b-a485-363d23a839e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:10:16.795889 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 566/1148] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:10:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:10:17.061395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcb052-3347-4e49-bafe-24d9806fd329 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:17.159953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcb052-3347-4e49-bafe-24d9806fd329 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:17.193203 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ed66f-784d-4f0e-abde-e564d52be411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:17.193812 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ed66f-784d-4f0e-abde-e564d52be411 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:17.194094 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ed66f-784d-4f0e-abde-e564d52be411 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:17.194370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ed66f-784d-4f0e-abde-e564d52be411 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:17.204130 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ed66f-784d-4f0e-abde-e564d52be411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:10:17.208947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-808ed66f-784d-4f0e-abde-e564d52be411 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:10:17.208947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:10:17.208947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:10:17.208947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:10:17.208947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:10:17.208947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:10:17.208947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:10:17.208947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:10:17.208947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:10:17.208947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:10:17.208947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:10:17.208947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:10:17.208947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:10:17.208947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:10:17.208947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:10:17.208947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:10:17.210061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:10:17.210061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:10:17.210061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:10:17.210061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:10:17.210061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:10:17.210061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:10:17.210061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:10:17.210061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:10:17.210061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:10:17.210061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:10:17.210061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:10:17.210061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:10:17.210061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:10:17.210061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:10:17.210061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:10:17.210061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:10:17.211274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:10:17.211274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:10:17.211274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:10:17.211274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:10:17.211274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:10:17.211274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:10:17.211274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:10:17.211274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:10:17.211274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:10:17.211274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:10:17.211274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:10:17.211274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:10:17.211274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 583/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:10:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:10:17.216024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd48adf-6f0e-479b-a485-363d23a839e4 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:17.310115 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd48adf-6f0e-479b-a485-363d23a839e4 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:17.335293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:17.343141 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca200e-b311-4e90-923b-cbc571a1fe48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:17.343770 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca200e-b311-4e90-923b-cbc571a1fe48 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:17.344049 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca200e-b311-4e90-923b-cbc571a1fe48 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:17.344317 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca200e-b311-4e90-923b-cbc571a1fe48 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:17.344966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca200e-b311-4e90-923b-cbc571a1fe48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:10:17.345264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca200e-b311-4e90-923b-cbc571a1fe48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:17.346643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca200e-b311-4e90-923b-cbc571a1fe48 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:17.354553 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 567/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:10:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:17.360389 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ed66f-784d-4f0e-abde-e564d52be411 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:17.408487 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:17.458720 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ed66f-784d-4f0e-abde-e564d52be411 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:17.494297 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c2987-973b-41d0-bc82-991c53b96c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:17.494909 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c2987-973b-41d0-bc82-991c53b96c76 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:17.495187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c2987-973b-41d0-bc82-991c53b96c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:17.495457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c2987-973b-41d0-bc82-991c53b96c76 None admin] PATH_INFO: `/v3/users/7bf0717e809d403e9712ab1de6ea5a62` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:17.496577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c2987-973b-41d0-bc82-991c53b96c76 None admin] RBAC: Authorizing `identity:get_user(user_id=7bf0717e809d403e9712ab1de6ea5a62)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:17.497990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c2987-973b-41d0-bc82-991c53b96c76 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:17.499563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 584/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:10:17 2022] GET /identity/v3/users/7bf0717e809d403e9712ab1de6ea5a62 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:17.505320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca200e-b311-4e90-923b-cbc571a1fe48 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:17.599796 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca200e-b311-4e90-923b-cbc571a1fe48 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:17.631108 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543b821-6403-459b-976d-3599371fbefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:17.631738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543b821-6403-459b-976d-3599371fbefa None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:17.632015 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543b821-6403-459b-976d-3599371fbefa None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:17.632283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543b821-6403-459b-976d-3599371fbefa None admin] PATH_INFO: `/v3/projects/f03af4c2207945a785107a7e3123ee97` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:17.633369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543b821-6403-459b-976d-3599371fbefa None admin] RBAC: Authorizing `identity:get_project(project_id=f03af4c2207945a785107a7e3123ee97)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:17.634778 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543b821-6403-459b-976d-3599371fbefa None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:17.636382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 568/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:10:17 2022] GET /identity/v3/projects/f03af4c2207945a785107a7e3123ee97 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:17.642187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c2987-973b-41d0-bc82-991c53b96c76 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:17.737699 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c2987-973b-41d0-bc82-991c53b96c76 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:17.753035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:17.770032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592ce08-7593-46de-82eb-e08da0aa6557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:17.770664 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592ce08-7593-46de-82eb-e08da0aa6557 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:17.770942 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592ce08-7593-46de-82eb-e08da0aa6557 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:17.771211 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592ce08-7593-46de-82eb-e08da0aa6557 None admin] PATH_INFO: `/v3/projects/f03af4c2207945a785107a7e3123ee97/users/7bf0717e809d403e9712ab1de6ea5a62/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:17.773099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592ce08-7593-46de-82eb-e08da0aa6557 None admin] RBAC: Authorizing `identity:create_grant(project_id=f03af4c2207945a785107a7e3123ee97, user_id=7bf0717e809d403e9712ab1de6ea5a62, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:17.774521 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592ce08-7593-46de-82eb-e08da0aa6557 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:17.788448 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 585/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:10:17 2022] PUT /identity/v3/projects/f03af4c2207945a785107a7e3123ee97/users/7bf0717e809d403e9712ab1de6ea5a62/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:10:19.907188 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a5236-d759-40db-b491-4654aeff4652 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:19.907775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a5236-d759-40db-b491-4654aeff4652 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:19.907775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a5236-d759-40db-b491-4654aeff4652 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:19.909082 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 569/1154] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:10:19.914165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae803e7f-4004-46ef-a03b-58664b87d8ee None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:19.914587 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae803e7f-4004-46ef-a03b-58664b87d8ee None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:19.914694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae803e7f-4004-46ef-a03b-58664b87d8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:19.932983 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ae803e7f-4004-46ef-a03b-58664b87d8ee None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:10:19.934151 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ae803e7f-4004-46ef-a03b-58664b87d8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:10:19.999392 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 586/1155] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:10:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:10:20.265367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a5236-d759-40db-b491-4654aeff4652 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:20.324270 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:20.366514 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a5236-d759-40db-b491-4654aeff4652 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:20.400831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd52a1-d904-446a-aa72-c236bce438a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:20.401443 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd52a1-d904-446a-aa72-c236bce438a8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:20.401723 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd52a1-d904-446a-aa72-c236bce438a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:20.401995 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd52a1-d904-446a-aa72-c236bce438a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:20.411982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd52a1-d904-446a-aa72-c236bce438a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:10:20.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-f6cd52a1-d904-446a-aa72-c236bce438a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:10:20.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:10:20.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:10:20.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:10:20.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:10:20.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:10:20.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:10:20.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:10:20.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:10:20.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:10:20.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:10:20.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:10:20.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:10:20.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:10:20.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:10:20.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:10:20.417910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:10:20.417910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:10:20.417910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:10:20.417910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:10:20.417910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:10:20.417910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:10:20.417910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:10:20.417910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:10:20.417910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:10:20.417910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:10:20.417910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:10:20.417910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:10:20.417910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:10:20.417910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:10:20.417910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:10:20.417910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:10:20.419132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:10:20.419132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:10:20.419132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:10:20.419132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:10:20.419132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:10:20.419132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:10:20.419132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:10:20.419132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:10:20.419132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:10:20.419132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:10:20.419132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:10:20.419132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:10:20.419132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 570/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:10:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:10:20.424404 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae803e7f-4004-46ef-a03b-58664b87d8ee None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:20.518937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae803e7f-4004-46ef-a03b-58664b87d8ee None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:20.525882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:20.554173 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b8cc3-8d2c-4824-b3b4-4409652e80f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:20.554764 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b8cc3-8d2c-4824-b3b4-4409652e80f2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:20.555046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b8cc3-8d2c-4824-b3b4-4409652e80f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:20.555321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b8cc3-8d2c-4824-b3b4-4409652e80f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:20.556008 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b8cc3-8d2c-4824-b3b4-4409652e80f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:10:20.556314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b8cc3-8d2c-4824-b3b4-4409652e80f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:20.557666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b8cc3-8d2c-4824-b3b4-4409652e80f2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:20.565462 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 587/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:10:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:20.571228 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd52a1-d904-446a-aa72-c236bce438a8 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:20.667797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd52a1-d904-446a-aa72-c236bce438a8 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:20.684627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:20.700923 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d24567-869b-4533-8b67-e607ef5dfdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:20.701545 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d24567-869b-4533-8b67-e607ef5dfdc3 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:20.701824 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d24567-869b-4533-8b67-e607ef5dfdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:20.702094 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d24567-869b-4533-8b67-e607ef5dfdc3 None admin] PATH_INFO: `/v3/users/7bf0717e809d403e9712ab1de6ea5a62` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:20.703191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d24567-869b-4533-8b67-e607ef5dfdc3 None admin] RBAC: Authorizing `identity:get_user(user_id=7bf0717e809d403e9712ab1de6ea5a62)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:20.704684 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d24567-869b-4533-8b67-e607ef5dfdc3 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:20.706231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 571/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:10:20 2022] GET /identity/v3/users/7bf0717e809d403e9712ab1de6ea5a62 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:20.712052 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b8cc3-8d2c-4824-b3b4-4409652e80f2 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:20.808164 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b8cc3-8d2c-4824-b3b4-4409652e80f2 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:20.839294 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa7b44-1da5-45d6-bb8e-f10ac59bb5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:20.839912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa7b44-1da5-45d6-bb8e-f10ac59bb5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:20.840193 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa7b44-1da5-45d6-bb8e-f10ac59bb5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:20.840463 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa7b44-1da5-45d6-bb8e-f10ac59bb5c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:20.873562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:20.905986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa7b44-1da5-45d6-bb8e-f10ac59bb5c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:20.907429 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa7b44-1da5-45d6-bb8e-f10ac59bb5c7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:21.001270 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 588/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:10:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:10:21.007220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d24567-869b-4533-8b67-e607ef5dfdc3 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:21.104291 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d24567-869b-4533-8b67-e607ef5dfdc3 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:21.135578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823792e2-ab84-4ee2-9044-a4a79712081d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:21.136187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823792e2-ab84-4ee2-9044-a4a79712081d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:21.136462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823792e2-ab84-4ee2-9044-a4a79712081d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:21.136770 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823792e2-ab84-4ee2-9044-a4a79712081d None admin] PATH_INFO: `/v3/projects/f03af4c2207945a785107a7e3123ee97` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:21.137861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823792e2-ab84-4ee2-9044-a4a79712081d None admin] RBAC: Authorizing `identity:get_project(project_id=f03af4c2207945a785107a7e3123ee97)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:21.139259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823792e2-ab84-4ee2-9044-a4a79712081d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:21.140863 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 572/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:10:21 2022] GET /identity/v3/projects/f03af4c2207945a785107a7e3123ee97 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:21.147326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa7b44-1da5-45d6-bb8e-f10ac59bb5c7 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:21.211238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:21.243163 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa7b44-1da5-45d6-bb8e-f10ac59bb5c7 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:21.276455 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69b1e0-644d-4397-bccd-9c5d11f9d325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:21.277046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69b1e0-644d-4397-bccd-9c5d11f9d325 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:21.277577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69b1e0-644d-4397-bccd-9c5d11f9d325 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:21.277846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69b1e0-644d-4397-bccd-9c5d11f9d325 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:21.278553 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69b1e0-644d-4397-bccd-9c5d11f9d325 None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=f03af4c2207945a785107a7e3123ee97, user.id=7bf0717e809d403e9712ab1de6ea5a62) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:10:21.278859 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69b1e0-644d-4397-bccd-9c5d11f9d325 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:21.280306 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69b1e0-644d-4397-bccd-9c5d11f9d325 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:21.285869 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 589/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:10:21 2022] GET /identity/v3/role_assignments?user.id=7bf0717e809d403e9712ab1de6ea5a62&scope.project.id=f03af4c2207945a785107a7e3123ee97&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:23.417747 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1c76e-2ffc-4634-8caa-94de7243b7e2 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:23.418292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1c76e-2ffc-4634-8caa-94de7243b7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:23.418292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1c76e-2ffc-4634-8caa-94de7243b7e2 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:23.419636 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 573/1162] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:10:23.424975 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227e4cb-12c7-43b6-b1e2-4d4799f047dd None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:23.425493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227e4cb-12c7-43b6-b1e2-4d4799f047dd None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:23.425865 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227e4cb-12c7-43b6-b1e2-4d4799f047dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:23.444666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3227e4cb-12c7-43b6-b1e2-4d4799f047dd None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:10:23.445860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3227e4cb-12c7-43b6-b1e2-4d4799f047dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:10:23.518033 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 590/1163] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:10:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:10:23.786585 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1c76e-2ffc-4634-8caa-94de7243b7e2 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:23.842253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:23.886986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1c76e-2ffc-4634-8caa-94de7243b7e2 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:23.919678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb68cd-b678-4470-aac1-99cbf62c1abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:23.920274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb68cd-b678-4470-aac1-99cbf62c1abf None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:23.920553 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb68cd-b678-4470-aac1-99cbf62c1abf None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:23.920824 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb68cd-b678-4470-aac1-99cbf62c1abf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:23.984058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb68cd-b678-4470-aac1-99cbf62c1abf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:23.985484 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb68cd-b678-4470-aac1-99cbf62c1abf None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:24.036421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:24.079977 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 574/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:10:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:10:24.085792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227e4cb-12c7-43b6-b1e2-4d4799f047dd None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:24.145623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:24.187289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227e4cb-12c7-43b6-b1e2-4d4799f047dd None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:24.221341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe3f3c-ff17-4514-87d0-866cf289366b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:24.221967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe3f3c-ff17-4514-87d0-866cf289366b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:24.222249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe3f3c-ff17-4514-87d0-866cf289366b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:24.222525 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe3f3c-ff17-4514-87d0-866cf289366b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:24.229292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe3f3c-ff17-4514-87d0-866cf289366b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:24.230734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe3f3c-ff17-4514-87d0-866cf289366b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-0dbe3f3c-ff17-4514-87d0-866cf289366b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:10:24.242696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:10:24.244383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:10:24.244383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:10:24.244383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:10:24.244383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:10:24.244383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:10:24.244383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:10:24.244383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:10:24.244383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:10:24.244383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:10:24.244383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:10:24.244383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:10:24.244383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:10:24.244383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:10:24.244383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:10:24.244383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:10:24.244383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:10:24.245350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:10:24.245350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:10:24.245350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:10:24.245350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:10:24.245350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:10:24.245350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:10:24.245350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:10:24.245350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:10:24.245350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:10:24.245350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:10:24.245350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 06 12:10:24.245350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:10:24.245350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 06 12:10:24.245350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:10:24.245350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 591/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Oct 6 12:10:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:10:24.251912 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb68cd-b678-4470-aac1-99cbf62c1abf None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:24.350987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb68cd-b678-4470-aac1-99cbf62c1abf None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:24.383940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e12b76-21b1-447e-a3e8-380a09d795ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:24.384546 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e12b76-21b1-447e-a3e8-380a09d795ec None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:24.384824 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e12b76-21b1-447e-a3e8-380a09d795ec None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:24.385096 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e12b76-21b1-447e-a3e8-380a09d795ec None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:24.385734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e12b76-21b1-447e-a3e8-380a09d795ec None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:10:24.386035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e12b76-21b1-447e-a3e8-380a09d795ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:24.387380 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e12b76-21b1-447e-a3e8-380a09d795ec None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:24.394891 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 575/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Oct 6 12:10:24 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:26.499097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff360da-bd5a-4b9f-821f-6d9393145cc9 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:26.499660 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff360da-bd5a-4b9f-821f-6d9393145cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:26.499660 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff360da-bd5a-4b9f-821f-6d9393145cc9 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:26.500964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 592/1167] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:10:26.506729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438ad24-ff0f-4956-9278-4686cb4f8db4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:26.507156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438ad24-ff0f-4956-9278-4686cb4f8db4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:26.507262 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438ad24-ff0f-4956-9278-4686cb4f8db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:26.525449 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7438ad24-ff0f-4956-9278-4686cb4f8db4 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:10:26.526617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7438ad24-ff0f-4956-9278-4686cb4f8db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:10:26.539358 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:26.590625 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 576/1168] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:10:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:10:26.861296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff360da-bd5a-4b9f-821f-6d9393145cc9 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:26.959887 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff360da-bd5a-4b9f-821f-6d9393145cc9 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:26.981654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:26.993651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae642e-2911-4688-970e-bd73ee227dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:26.994252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae642e-2911-4688-970e-bd73ee227dc8 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:26.994535 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae642e-2911-4688-970e-bd73ee227dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:26.994807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae642e-2911-4688-970e-bd73ee227dc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:26.995459 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae642e-2911-4688-970e-bd73ee227dc8 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:26.996850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae642e-2911-4688-970e-bd73ee227dc8 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:27.012209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 593/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Thu Oct 6 12:10:26 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:10:29.144556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebf4b7-4d8f-43f3-8997-4d547186873f None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:29.145114 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebf4b7-4d8f-43f3-8997-4d547186873f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:29.145114 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebf4b7-4d8f-43f3-8997-4d547186873f None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:29.146412 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 577/1170] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:10:29.151499 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc2bdb-5e78-4ebe-b96d-087a14ade550 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:29.151926 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc2bdb-5e78-4ebe-b96d-087a14ade550 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:29.152324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc2bdb-5e78-4ebe-b96d-087a14ade550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:29.170557 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-57bc2bdb-5e78-4ebe-b96d-087a14ade550 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:10:29.171756 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-57bc2bdb-5e78-4ebe-b96d-087a14ade550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:10:29.236463 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 594/1171] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:10:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:10:29.499076 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebf4b7-4d8f-43f3-8997-4d547186873f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:29.595826 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebf4b7-4d8f-43f3-8997-4d547186873f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:29.628370 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b728dfb-05f5-4fd6-98bf-a68c254d0fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:29.628975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b728dfb-05f5-4fd6-98bf-a68c254d0fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:29.629250 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b728dfb-05f5-4fd6-98bf-a68c254d0fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:29.629519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b728dfb-05f5-4fd6-98bf-a68c254d0fb8 None admin] PATH_INFO: `/v3/domains/149a2ac1cf774ffc92f9489b5e8f1374` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:29.639279 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b728dfb-05f5-4fd6-98bf-a68c254d0fb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=149a2ac1cf774ffc92f9489b5e8f1374)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:29.640740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b728dfb-05f5-4fd6-98bf-a68c254d0fb8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:29.642293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 578/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Thu Oct 6 12:10:29 2022] GET /identity/v3/domains/149a2ac1cf774ffc92f9489b5e8f1374 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:29.648221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc2bdb-5e78-4ebe-b96d-087a14ade550 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:29.743841 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc2bdb-5e78-4ebe-b96d-087a14ade550 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:29.756932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:29.778424 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480e1c8-f104-4a60-b2d2-271641fee198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:29.779019 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480e1c8-f104-4a60-b2d2-271641fee198 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:29.779300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480e1c8-f104-4a60-b2d2-271641fee198 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:29.779607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480e1c8-f104-4a60-b2d2-271641fee198 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:29.780267 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480e1c8-f104-4a60-b2d2-271641fee198 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:29.781682 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480e1c8-f104-4a60-b2d2-271641fee198 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:29.808248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 595/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Oct 6 12:10:29 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:10:31.934986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a135a-23d6-42bc-9bf0-2fa8470c859b None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:31.935579 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a135a-23d6-42bc-9bf0-2fa8470c859b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:31.935579 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a135a-23d6-42bc-9bf0-2fa8470c859b None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:31.936996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 579/1174] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:10:31.942265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dae6bc-9855-4fd8-bcd2-0875ba648672 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:31.942635 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dae6bc-9855-4fd8-bcd2-0875ba648672 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:31.942815 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dae6bc-9855-4fd8-bcd2-0875ba648672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:31.961631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-14dae6bc-9855-4fd8-bcd2-0875ba648672 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:10:31.962889 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-14dae6bc-9855-4fd8-bcd2-0875ba648672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:10:32.028172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 596/1175] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:10:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:10:32.295293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a135a-23d6-42bc-9bf0-2fa8470c859b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:32.307017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:32.393839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a135a-23d6-42bc-9bf0-2fa8470c859b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:32.427188 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4280160-4f0a-4871-b9b7-c21e99e19b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:32.427832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4280160-4f0a-4871-b9b7-c21e99e19b69 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:32.428111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4280160-4f0a-4871-b9b7-c21e99e19b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:32.428382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4280160-4f0a-4871-b9b7-c21e99e19b69 None admin] PATH_INFO: `/v3/domains/149a2ac1cf774ffc92f9489b5e8f1374` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:32.429483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4280160-4f0a-4871-b9b7-c21e99e19b69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=149a2ac1cf774ffc92f9489b5e8f1374)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:32.430910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4280160-4f0a-4871-b9b7-c21e99e19b69 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:32.432486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 580/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Thu Oct 6 12:10:32 2022] GET /identity/v3/domains/149a2ac1cf774ffc92f9489b5e8f1374 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:32.438221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dae6bc-9855-4fd8-bcd2-0875ba648672 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:32.528234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:32.536463 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dae6bc-9855-4fd8-bcd2-0875ba648672 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:32.569094 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87502e-71b9-4929-a536-028f4deb549e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:32.569677 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87502e-71b9-4929-a536-028f4deb549e None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:32.569951 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87502e-71b9-4929-a536-028f4deb549e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:32.570219 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87502e-71b9-4929-a536-028f4deb549e None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:32.570846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87502e-71b9-4929-a536-028f4deb549e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:32.572295 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87502e-71b9-4929-a536-028f4deb549e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:32.593356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 597/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:10:32 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:10:34.761142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93585f-c363-4af2-9684-81197b10e201 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:34.761707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93585f-c363-4af2-9684-81197b10e201 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:34.761707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93585f-c363-4af2-9684-81197b10e201 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:34.763007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 581/1178] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:10:34.768802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee006da-1f63-4aad-98de-3c9158b7e2c2 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:34.769209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee006da-1f63-4aad-98de-3c9158b7e2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:34.769348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee006da-1f63-4aad-98de-3c9158b7e2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:34.787909 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fee006da-1f63-4aad-98de-3c9158b7e2c2 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:10:34.789083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fee006da-1f63-4aad-98de-3c9158b7e2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:10:34.852542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 598/1179] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:10:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:10:35.120013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93585f-c363-4af2-9684-81197b10e201 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:35.195754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:35.218567 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93585f-c363-4af2-9684-81197b10e201 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:35.251771 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6aa187-555c-452d-a42e-781eb48abb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:35.252404 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6aa187-555c-452d-a42e-781eb48abb0f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:35.252682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6aa187-555c-452d-a42e-781eb48abb0f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:35.252952 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6aa187-555c-452d-a42e-781eb48abb0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:35.262795 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6aa187-555c-452d-a42e-781eb48abb0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:10:35.267586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-4b6aa187-555c-452d-a42e-781eb48abb0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:10:35.267586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:10:35.267586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:10:35.267586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:10:35.267586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:10:35.267586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:10:35.267586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:10:35.267586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:10:35.267586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:10:35.267586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:10:35.267586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:10:35.267586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:10:35.267586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:10:35.267586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:10:35.267586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:10:35.267586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:10:35.268722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:10:35.268722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:10:35.268722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:10:35.268722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:10:35.268722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:10:35.268722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:10:35.268722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:10:35.268722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:10:35.268722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:10:35.268722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:10:35.268722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:10:35.268722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:10:35.268722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:10:35.268722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:10:35.268722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:10:35.268722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:10:35.269776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:10:35.269776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:10:35.269776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:10:35.269776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:10:35.269776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:10:35.269776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:10:35.269776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:10:35.269776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:10:35.269776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:10:35.269776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:10:35.269776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:10:35.269776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:10:35.269776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 582/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:10:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:10:35.274666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee006da-1f63-4aad-98de-3c9158b7e2c2 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:35.351039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:35.370296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee006da-1f63-4aad-98de-3c9158b7e2c2 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:35.402343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8a7e1-a215-497e-bdac-b71bff9a9f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:35.402929 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8a7e1-a215-497e-bdac-b71bff9a9f5e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:35.403203 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8a7e1-a215-497e-bdac-b71bff9a9f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:35.403470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8a7e1-a215-497e-bdac-b71bff9a9f5e None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:35.404141 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8a7e1-a215-497e-bdac-b71bff9a9f5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:10:35.404440 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8a7e1-a215-497e-bdac-b71bff9a9f5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:35.405786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8a7e1-a215-497e-bdac-b71bff9a9f5e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:35.413937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 599/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:10:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:35.419749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6aa187-555c-452d-a42e-781eb48abb0f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:35.514900 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6aa187-555c-452d-a42e-781eb48abb0f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:35.548265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b057e-2e38-4b1f-a686-84e82a64037a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:35.548877 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b057e-2e38-4b1f-a686-84e82a64037a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:35.549154 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b057e-2e38-4b1f-a686-84e82a64037a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:35.549421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b057e-2e38-4b1f-a686-84e82a64037a None admin] PATH_INFO: `/v3/users/2d31b8fab07743c69630823458c71293` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:35.552011 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:35.562319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b057e-2e38-4b1f-a686-84e82a64037a None admin] RBAC: Authorizing `identity:get_user(user_id=2d31b8fab07743c69630823458c71293)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:35.563824 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b057e-2e38-4b1f-a686-84e82a64037a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:35.565415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 583/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:10:35 2022] GET /identity/v3/users/2d31b8fab07743c69630823458c71293 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:35.571335 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8a7e1-a215-497e-bdac-b71bff9a9f5e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:35.668098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8a7e1-a215-497e-bdac-b71bff9a9f5e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:35.699426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26a1a8-7573-4b7d-bee0-1d0569d403bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:35.700044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26a1a8-7573-4b7d-bee0-1d0569d403bf None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:35.700330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26a1a8-7573-4b7d-bee0-1d0569d403bf None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:35.700602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26a1a8-7573-4b7d-bee0-1d0569d403bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:35.712305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:35.765248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26a1a8-7573-4b7d-bee0-1d0569d403bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:35.767198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de26a1a8-7573-4b7d-bee0-1d0569d403bf None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:35.860373 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 600/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:10:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:10:35.866181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b057e-2e38-4b1f-a686-84e82a64037a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:35.961259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b057e-2e38-4b1f-a686-84e82a64037a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:35.993678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89098297-0d5d-4661-af1c-3e6065259c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:35.994289 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89098297-0d5d-4661-af1c-3e6065259c23 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:35.994568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89098297-0d5d-4661-af1c-3e6065259c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:35.994837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89098297-0d5d-4661-af1c-3e6065259c23 None admin] PATH_INFO: `/v3/projects/0a4e9339808349cfb4ff74750170cee7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:35.995949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89098297-0d5d-4661-af1c-3e6065259c23 None admin] RBAC: Authorizing `identity:get_project(project_id=0a4e9339808349cfb4ff74750170cee7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:35.997357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89098297-0d5d-4661-af1c-3e6065259c23 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:35.998914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 584/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:10:35 2022] GET /identity/v3/projects/0a4e9339808349cfb4ff74750170cee7 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:36.004942 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26a1a8-7573-4b7d-bee0-1d0569d403bf None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:36.046156 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:36.100738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26a1a8-7573-4b7d-bee0-1d0569d403bf None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:36.133658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5d6ea-3421-4cb4-a27a-014ea1b8492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:36.134241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5d6ea-3421-4cb4-a27a-014ea1b8492a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:36.134521 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5d6ea-3421-4cb4-a27a-014ea1b8492a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:36.134789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5d6ea-3421-4cb4-a27a-014ea1b8492a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:36.135497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5d6ea-3421-4cb4-a27a-014ea1b8492a None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=0a4e9339808349cfb4ff74750170cee7, user.id=2d31b8fab07743c69630823458c71293) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:10:36.135831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5d6ea-3421-4cb4-a27a-014ea1b8492a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:36.137235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5d6ea-3421-4cb4-a27a-014ea1b8492a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:36.142328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 601/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:10:36 2022] GET /identity/v3/role_assignments?user.id=2d31b8fab07743c69630823458c71293&scope.project.id=0a4e9339808349cfb4ff74750170cee7&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:38.253464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cb8d7-9af9-4c60-b9af-8b5c91cd3e3d None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:38.253897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cb8d7-9af9-4c60-b9af-8b5c91cd3e3d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:38.254018 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cb8d7-9af9-4c60-b9af-8b5c91cd3e3d None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:38.255318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 585/1186] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:10:38.260580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35e2b0-b1d5-4da9-a3cf-2c32d68da276 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:38.260989 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35e2b0-b1d5-4da9-a3cf-2c32d68da276 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:38.261126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35e2b0-b1d5-4da9-a3cf-2c32d68da276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:38.279216 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-4e35e2b0-b1d5-4da9-a3cf-2c32d68da276 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:10:38.280433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4e35e2b0-b1d5-4da9-a3cf-2c32d68da276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:10:38.344288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 602/1187] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:10:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:10:38.614756 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cb8d7-9af9-4c60-b9af-8b5c91cd3e3d None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:38.674744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:38.712076 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cb8d7-9af9-4c60-b9af-8b5c91cd3e3d None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:38.744292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83733c58-34e8-4251-89a2-701a0d3c6f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:38.744901 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83733c58-34e8-4251-89a2-701a0d3c6f54 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:38.745177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83733c58-34e8-4251-89a2-701a0d3c6f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:38.745445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83733c58-34e8-4251-89a2-701a0d3c6f54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:38.755676 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83733c58-34e8-4251-89a2-701a0d3c6f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:10:38.760345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-83733c58-34e8-4251-89a2-701a0d3c6f54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:10:38.760345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:10:38.760345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:10:38.760345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:10:38.760345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:10:38.760345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:10:38.760345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:10:38.760345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:10:38.760345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:10:38.760345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:10:38.760345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:10:38.760345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:10:38.760345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:10:38.760345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:10:38.760345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:10:38.760345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:10:38.761463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:10:38.761463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:10:38.761463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:10:38.761463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:10:38.761463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:10:38.761463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:10:38.761463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:10:38.761463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:10:38.761463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:10:38.761463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:10:38.761463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:10:38.761463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:10:38.761463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:10:38.761463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:10:38.761463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:10:38.761463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:10:38.762374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:10:38.762374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:10:38.762374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:10:38.762374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:10:38.762374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:10:38.762374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:10:38.762374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:10:38.762374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:10:38.762374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:10:38.762374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:10:38.762374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:10:38.762374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:10:38.762374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 586/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:10:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:10:38.767330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35e2b0-b1d5-4da9-a3cf-2c32d68da276 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:38.806042 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:38.867093 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35e2b0-b1d5-4da9-a3cf-2c32d68da276 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:38.899677 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30b869-3f4e-4305-9660-01b872d38b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:38.900282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30b869-3f4e-4305-9660-01b872d38b30 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:38.900561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30b869-3f4e-4305-9660-01b872d38b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:38.900831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30b869-3f4e-4305-9660-01b872d38b30 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:38.901488 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30b869-3f4e-4305-9660-01b872d38b30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:10:38.902043 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30b869-3f4e-4305-9660-01b872d38b30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:38.903398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30b869-3f4e-4305-9660-01b872d38b30 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:38.911790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 603/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:10:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:38.917674 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83733c58-34e8-4251-89a2-701a0d3c6f54 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:39.012245 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83733c58-34e8-4251-89a2-701a0d3c6f54 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:39.034049 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:39.045497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742ea0a-cb5a-48f7-a6bf-d592b88121c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:39.046101 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742ea0a-cb5a-48f7-a6bf-d592b88121c9 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:39.046376 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742ea0a-cb5a-48f7-a6bf-d592b88121c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:39.046647 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742ea0a-cb5a-48f7-a6bf-d592b88121c9 None admin] PATH_INFO: `/v3/users/2d31b8fab07743c69630823458c71293` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:39.048321 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742ea0a-cb5a-48f7-a6bf-d592b88121c9 None admin] RBAC: Authorizing `identity:get_user(user_id=2d31b8fab07743c69630823458c71293)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:39.049729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742ea0a-cb5a-48f7-a6bf-d592b88121c9 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:39.051237 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 587/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:10:38 2022] GET /identity/v3/users/2d31b8fab07743c69630823458c71293 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:39.056837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30b869-3f4e-4305-9660-01b872d38b30 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:39.164058 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:39.168062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30b869-3f4e-4305-9660-01b872d38b30 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:39.201259 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff2294-dab9-4b6e-a781-bd2e311b8f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:39.201867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff2294-dab9-4b6e-a781-bd2e311b8f5b None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:39.202145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff2294-dab9-4b6e-a781-bd2e311b8f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:39.202416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff2294-dab9-4b6e-a781-bd2e311b8f5b None admin] PATH_INFO: `/v3/projects/0a4e9339808349cfb4ff74750170cee7` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:39.204116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff2294-dab9-4b6e-a781-bd2e311b8f5b None admin] RBAC: Authorizing `identity:get_project(project_id=0a4e9339808349cfb4ff74750170cee7)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:39.205523 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff2294-dab9-4b6e-a781-bd2e311b8f5b None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:39.207118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 604/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:10:39 2022] GET /identity/v3/projects/0a4e9339808349cfb4ff74750170cee7 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:39.213045 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742ea0a-cb5a-48f7-a6bf-d592b88121c9 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:39.310884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742ea0a-cb5a-48f7-a6bf-d592b88121c9 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:39.343279 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec99a57-7057-464d-b30e-eb74c8970b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:39.343950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec99a57-7057-464d-b30e-eb74c8970b1c None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:39.344232 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec99a57-7057-464d-b30e-eb74c8970b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:39.344504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec99a57-7057-464d-b30e-eb74c8970b1c None admin] PATH_INFO: `/v3/projects/0a4e9339808349cfb4ff74750170cee7/users/2d31b8fab07743c69630823458c71293/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:39.346376 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec99a57-7057-464d-b30e-eb74c8970b1c None admin] RBAC: Authorizing `identity:create_grant(project_id=0a4e9339808349cfb4ff74750170cee7, user_id=2d31b8fab07743c69630823458c71293, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:39.347821 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec99a57-7057-464d-b30e-eb74c8970b1c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:39.360615 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 588/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:10:39 2022] PUT /identity/v3/projects/0a4e9339808349cfb4ff74750170cee7/users/2d31b8fab07743c69630823458c71293/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:10:41.444001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d1c69-7ece-40e8-b276-e165bd55e3a8 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:41.444551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d1c69-7ece-40e8-b276-e165bd55e3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:41.444551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d1c69-7ece-40e8-b276-e165bd55e3a8 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:41.445875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 605/1193] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:10:41.450949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2784a41-8356-4e62-8cbd-ec8b162182f7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:41.451392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2784a41-8356-4e62-8cbd-ec8b162182f7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:41.451604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2784a41-8356-4e62-8cbd-ec8b162182f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:41.469948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c2784a41-8356-4e62-8cbd-ec8b162182f7 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:10:41.471144 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c2784a41-8356-4e62-8cbd-ec8b162182f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:10:41.515032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:41.536857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 589/1194] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:10:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:10:41.805387 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d1c69-7ece-40e8-b276-e165bd55e3a8 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:41.904053 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d1c69-7ece-40e8-b276-e165bd55e3a8 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:41.935478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e79aee-74ff-49a4-a4dc-81a244e54a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:41.936114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e79aee-74ff-49a4-a4dc-81a244e54a8e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:41.936395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e79aee-74ff-49a4-a4dc-81a244e54a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:41.936667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e79aee-74ff-49a4-a4dc-81a244e54a8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:41.947225 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e79aee-74ff-49a4-a4dc-81a244e54a8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:10:41.952296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-47e79aee-74ff-49a4-a4dc-81a244e54a8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:10:41.952296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:10:41.952296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:10:41.952296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:10:41.952296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:10:41.952296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:10:41.952296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:10:41.952296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:10:41.952296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:10:41.952296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:10:41.952296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:10:41.952296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:10:41.952296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:10:41.952296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:10:41.952296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:10:41.952296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:10:41.954159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:10:41.954159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:10:41.954159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:10:41.954159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:10:41.954159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:10:41.954159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:10:41.954159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:10:41.954159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:10:41.954159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:10:41.954159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:10:41.954159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:10:41.954159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:10:41.954159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:10:41.954159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:10:41.954159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:10:41.954159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:10:41.955761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:10:41.955761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:10:41.955761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:10:41.955761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:10:41.955761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:10:41.955761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:10:41.955761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:10:41.955761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:10:41.955761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:10:41.955761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:10:41.955761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:10:41.955761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:10:41.955761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 606/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:10:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:10:41.960011 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2784a41-8356-4e62-8cbd-ec8b162182f7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:42.054060 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2784a41-8356-4e62-8cbd-ec8b162182f7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:42.086687 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b115c0-6260-4c82-abf2-22d49e369e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:42.087224 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b115c0-6260-4c82-abf2-22d49e369e55 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:42.087527 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b115c0-6260-4c82-abf2-22d49e369e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:42.087802 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b115c0-6260-4c82-abf2-22d49e369e55 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:42.088455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b115c0-6260-4c82-abf2-22d49e369e55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:10:42.088754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b115c0-6260-4c82-abf2-22d49e369e55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:42.090110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b115c0-6260-4c82-abf2-22d49e369e55 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:42.097761 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 590/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:10:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:42.103394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e79aee-74ff-49a4-a4dc-81a244e54a8e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:42.130350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:42.199104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e79aee-74ff-49a4-a4dc-81a244e54a8e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:42.230874 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62571f23-1ef3-40d9-8b6f-e918a13aac08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:42.231483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62571f23-1ef3-40d9-8b6f-e918a13aac08 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:42.231791 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62571f23-1ef3-40d9-8b6f-e918a13aac08 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:42.232062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62571f23-1ef3-40d9-8b6f-e918a13aac08 None admin] PATH_INFO: `/v3/users/2d31b8fab07743c69630823458c71293` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:42.233160 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62571f23-1ef3-40d9-8b6f-e918a13aac08 None admin] RBAC: Authorizing `identity:get_user(user_id=2d31b8fab07743c69630823458c71293)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:42.234635 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62571f23-1ef3-40d9-8b6f-e918a13aac08 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:42.236239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 607/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Oct 6 12:10:42 2022] GET /identity/v3/users/2d31b8fab07743c69630823458c71293 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:42.241955 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b115c0-6260-4c82-abf2-22d49e369e55 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:42.284574 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:42.339375 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b115c0-6260-4c82-abf2-22d49e369e55 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:42.370979 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb16a4a-ef07-4500-b374-35aff161639c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:42.371600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb16a4a-ef07-4500-b374-35aff161639c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:42.371879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb16a4a-ef07-4500-b374-35aff161639c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:42.372145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb16a4a-ef07-4500-b374-35aff161639c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:42.435136 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb16a4a-ef07-4500-b374-35aff161639c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:42.436603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb16a4a-ef07-4500-b374-35aff161639c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:42.475717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:42.530736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 591/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:10:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:10:42.536855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62571f23-1ef3-40d9-8b6f-e918a13aac08 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:42.627728 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:42.634413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62571f23-1ef3-40d9-8b6f-e918a13aac08 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:42.666372 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953a4bf-d586-417f-9c7e-1a8cd51151ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:42.666976 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953a4bf-d586-417f-9c7e-1a8cd51151ce None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:42.667251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953a4bf-d586-417f-9c7e-1a8cd51151ce None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:42.667545 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953a4bf-d586-417f-9c7e-1a8cd51151ce None admin] PATH_INFO: `/v3/projects/0a4e9339808349cfb4ff74750170cee7` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:42.668639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b953a4bf-d586-417f-9c7e-1a8cd51151ce None admin] RBAC: Authorizing `identity:get_project(project_id=0a4e9339808349cfb4ff74750170cee7)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:42.670044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b953a4bf-d586-417f-9c7e-1a8cd51151ce None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:42.671640 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 608/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Oct 6 12:10:42 2022] GET /identity/v3/projects/0a4e9339808349cfb4ff74750170cee7 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:42.678174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb16a4a-ef07-4500-b374-35aff161639c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:10:42.771172 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb16a4a-ef07-4500-b374-35aff161639c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:10:42.804490 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8aeac6-f6dd-44ec-b60c-6e0df95a7f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:10:42.805077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8aeac6-f6dd-44ec-b60c-6e0df95a7f41 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:10:42.805354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8aeac6-f6dd-44ec-b60c-6e0df95a7f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:10:42.805624 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8aeac6-f6dd-44ec-b60c-6e0df95a7f41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:10:42.806326 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8aeac6-f6dd-44ec-b60c-6e0df95a7f41 None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=0a4e9339808349cfb4ff74750170cee7, user.id=2d31b8fab07743c69630823458c71293) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:10:42.806628 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8aeac6-f6dd-44ec-b60c-6e0df95a7f41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:10:42.808063 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8aeac6-f6dd-44ec-b60c-6e0df95a7f41 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:10:42.813596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 592/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:10:42 2022] GET /identity/v3/role_assignments?user.id=2d31b8fab07743c69630823458c71293&scope.project.id=0a4e9339808349cfb4ff74750170cee7&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:10:52.662441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=19 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:10:52.811044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:37.804847 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c98a6a-d27a-48ca-858e-2ece8befc9e5 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:37.805452 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c98a6a-d27a-48ca-858e-2ece8befc9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:37.805452 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c98a6a-d27a-48ca-858e-2ece8befc9e5 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:37.806902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 609/1201] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:11:37.812872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f824349-2ad8-43ab-8f2c-2fab83bc542e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:37.813402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f824349-2ad8-43ab-8f2c-2fab83bc542e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:37.813402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f824349-2ad8-43ab-8f2c-2fab83bc542e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:37.833559 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-5f824349-2ad8-43ab-8f2c-2fab83bc542e None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:11:37.834846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5f824349-2ad8-43ab-8f2c-2fab83bc542e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:11:37.919760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 593/1202] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:11:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:11:38.201049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c98a6a-d27a-48ca-858e-2ece8befc9e5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:38.335721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c98a6a-d27a-48ca-858e-2ece8befc9e5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:38.373030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca9f20-491f-4463-8c19-2bf3557b1821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:38.375904 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca9f20-491f-4463-8c19-2bf3557b1821 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:38.376213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca9f20-491f-4463-8c19-2bf3557b1821 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:38.376458 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca9f20-491f-4463-8c19-2bf3557b1821 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:38.388423 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca9f20-491f-4463-8c19-2bf3557b1821 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:38.390059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca9f20-491f-4463-8c19-2bf3557b1821 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:38.406560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-9eca9f20-491f-4463-8c19-2bf3557b1821 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:38.406560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:38.406560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:38.406560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:38.406560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:38.406560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:38.406560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:38.406560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:38.406560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:38.406560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:38.406560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:38.406560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:38.406560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:11:38.406560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:11:38.406560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:11:38.406560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:38.411850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:11:38.413015 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:11:38.413015 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:38.413015 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:11:38.413015 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 610/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:11:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:38.414672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f824349-2ad8-43ab-8f2c-2fab83bc542e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:38.543918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f824349-2ad8-43ab-8f2c-2fab83bc542e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:38.575678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:38.583003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcb0af-1a24-4bd2-9924-bf6dddce569b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:38.583638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcb0af-1a24-4bd2-9924-bf6dddce569b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:38.584091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcb0af-1a24-4bd2-9924-bf6dddce569b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:38.584473 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcb0af-1a24-4bd2-9924-bf6dddce569b None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:38.585084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcb0af-1a24-4bd2-9924-bf6dddce569b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:38.585406 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcb0af-1a24-4bd2-9924-bf6dddce569b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:38.586888 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcb0af-1a24-4bd2-9924-bf6dddce569b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:38.604647 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 594/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:38.605807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca9f20-491f-4463-8c19-2bf3557b1821 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:38.667043 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:38.719981 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca9f20-491f-4463-8c19-2bf3557b1821 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:38.759477 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce197d05-080d-460d-9ade-ea57afe4b1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:38.760216 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce197d05-080d-460d-9ade-ea57afe4b1f9 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:38.760994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce197d05-080d-460d-9ade-ea57afe4b1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:38.761339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce197d05-080d-460d-9ade-ea57afe4b1f9 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:38.762067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce197d05-080d-460d-9ade-ea57afe4b1f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:38.763604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce197d05-080d-460d-9ade-ea57afe4b1f9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:38.803814 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 611/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Oct 6 12:11:38 2022] POST /identity/v3/users => generated 251 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:11:41.177921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff427e74-1c94-4a06-a4ea-7a3d6b78d228 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:41.178519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff427e74-1c94-4a06-a4ea-7a3d6b78d228 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:41.178519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff427e74-1c94-4a06-a4ea-7a3d6b78d228 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:41.179836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 595/1206] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:11:41.185793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85723940-0588-4430-91ba-d26c08beccd5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:41.187835 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85723940-0588-4430-91ba-d26c08beccd5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:41.187835 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85723940-0588-4430-91ba-d26c08beccd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:41.206069 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-85723940-0588-4430-91ba-d26c08beccd5 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:11:41.207311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-85723940-0588-4430-91ba-d26c08beccd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:11:41.284308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 612/1207] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:11:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:11:41.552431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff427e74-1c94-4a06-a4ea-7a3d6b78d228 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:41.670200 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff427e74-1c94-4a06-a4ea-7a3d6b78d228 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:41.711437 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac587a-8123-438b-88f4-a7c0af4b9837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:41.712125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac587a-8123-438b-88f4-a7c0af4b9837 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:41.712452 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac587a-8123-438b-88f4-a7c0af4b9837 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:41.712873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac587a-8123-438b-88f4-a7c0af4b9837 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:41.724319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac587a-8123-438b-88f4-a7c0af4b9837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:11:41.730940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-d6ac587a-8123-438b-88f4-a7c0af4b9837 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:11:41.730940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:41.730940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:41.730940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:41.730940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:41.730940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:41.730940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:41.730940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:41.730940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:41.730940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:41.730940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:41.730940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:41.730940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:11:41.730940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:11:41.730940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:11:41.730940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:11:41.733596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:11:41.733596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:11:41.733596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:11:41.733596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:11:41.733596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:41.733596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:41.733596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:41.733596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:41.733596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:41.733596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:41.733596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:41.733596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:41.733596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:41.733596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:41.733596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:41.733596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:41.734513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:41.734513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:41.734513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:41.734513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:41.734513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:11:41.734513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:11:41.734513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:11:41.734513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:11:41.734513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:11:41.734513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:11:41.734513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:11:41.734513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:11:41.734513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 596/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:11:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:41.739083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85723940-0588-4430-91ba-d26c08beccd5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:41.787126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:41.856252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85723940-0588-4430-91ba-d26c08beccd5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:41.896150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6d688-5ed6-42aa-ae70-5b940991131e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:41.896983 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6d688-5ed6-42aa-ae70-5b940991131e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:41.897272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6d688-5ed6-42aa-ae70-5b940991131e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:41.897549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6d688-5ed6-42aa-ae70-5b940991131e None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:41.898226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa6d688-5ed6-42aa-ae70-5b940991131e None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:41.898538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa6d688-5ed6-42aa-ae70-5b940991131e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:41.900193 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa6d688-5ed6-42aa-ae70-5b940991131e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:41.910253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 613/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:11:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:41.917921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac587a-8123-438b-88f4-a7c0af4b9837 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:41.982215 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:42.047283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac587a-8123-438b-88f4-a7c0af4b9837 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:42.087957 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c609906-0c1c-45a0-9710-577e6903ac6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:42.089455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c609906-0c1c-45a0-9710-577e6903ac6f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:42.089697 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c609906-0c1c-45a0-9710-577e6903ac6f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:42.089973 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c609906-0c1c-45a0-9710-577e6903ac6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:42.101675 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c609906-0c1c-45a0-9710-577e6903ac6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:42.103224 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c609906-0c1c-45a0-9710-577e6903ac6f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:42.120299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-8c609906-0c1c-45a0-9710-577e6903ac6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:42.120299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:42.120299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:42.120299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:42.120299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:42.120299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:42.120299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:42.120299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:42.120299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:42.120299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:42.120299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:42.120299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:42.120299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:11:42.120299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:11:42.120299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:11:42.120299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:42.126128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:11:42.127131 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:11:42.127131 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:42.127131 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:11:42.127131 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 597/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:11:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:42.127392 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6d688-5ed6-42aa-ae70-5b940991131e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:42.236831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:42.241322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6d688-5ed6-42aa-ae70-5b940991131e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:42.284645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22776646-8884-4226-8294-ee91cec1ff37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:42.285411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22776646-8884-4226-8294-ee91cec1ff37 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:42.285722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22776646-8884-4226-8294-ee91cec1ff37 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:42.286064 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22776646-8884-4226-8294-ee91cec1ff37 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:42.286776 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22776646-8884-4226-8294-ee91cec1ff37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:42.287146 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22776646-8884-4226-8294-ee91cec1ff37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:42.288663 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22776646-8884-4226-8294-ee91cec1ff37 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:42.302646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 614/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:11:42.308258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c609906-0c1c-45a0-9710-577e6903ac6f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:42.424078 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c609906-0c1c-45a0-9710-577e6903ac6f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:42.425697 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:42.465788 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865eec8-1bbb-46a5-9b1b-4ec909c8d2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:42.466632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865eec8-1bbb-46a5-9b1b-4ec909c8d2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:42.467149 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865eec8-1bbb-46a5-9b1b-4ec909c8d2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:42.467695 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865eec8-1bbb-46a5-9b1b-4ec909c8d2b6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:42.476833 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865eec8-1bbb-46a5-9b1b-4ec909c8d2b6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:42.479000 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865eec8-1bbb-46a5-9b1b-4ec909c8d2b6 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:42.496240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-c865eec8-1bbb-46a5-9b1b-4ec909c8d2b6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 12:11:42.496240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:42.496240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:42.496240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:42.496240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:42.496240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:42.496240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:42.496240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:42.496240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:42.496240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:42.496240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:42.496240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:42.496240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:11:42.496240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:11:42.496240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:11:42.496240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:42.497569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:42.498755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:42.498755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:42.498755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:11:42.498755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:11:42.498755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:11:42.498755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:11:42.498755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:11:42.498755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:11:42.498755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 12:11:42.498755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:11:42.504662 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 598/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:11:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:42.505166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22776646-8884-4226-8294-ee91cec1ff37 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:42.624260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22776646-8884-4226-8294-ee91cec1ff37 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:42.665817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fac9c3-7314-423e-9653-b90e318c8bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:42.666424 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fac9c3-7314-423e-9653-b90e318c8bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:42.666693 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fac9c3-7314-423e-9653-b90e318c8bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:42.667522 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fac9c3-7314-423e-9653-b90e318c8bf2 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:42.668253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fac9c3-7314-423e-9653-b90e318c8bf2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:42.668559 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fac9c3-7314-423e-9653-b90e318c8bf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:42.669976 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fac9c3-7314-423e-9653-b90e318c8bf2 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:42.689903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 615/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Oct 6 12:11:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:11:42.696026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865eec8-1bbb-46a5-9b1b-4ec909c8d2b6 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:42.808715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865eec8-1bbb-46a5-9b1b-4ec909c8d2b6 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:42.846776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff498f-ead3-4992-a0d7-a796a7466433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:42.847381 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff498f-ead3-4992-a0d7-a796a7466433 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:42.847689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff498f-ead3-4992-a0d7-a796a7466433 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:42.847957 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff498f-ead3-4992-a0d7-a796a7466433 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:42.879283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:42.921405 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff498f-ead3-4992-a0d7-a796a7466433 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:42.922885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff498f-ead3-4992-a0d7-a796a7466433 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:43.049261 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 599/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:11:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:11:43.055090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fac9c3-7314-423e-9653-b90e318c8bf2 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:43.075172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:43.171456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fac9c3-7314-423e-9653-b90e318c8bf2 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:43.209486 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624a027-1d5c-4de0-bce0-be1e0b54ea06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:43.210114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624a027-1d5c-4de0-bce0-be1e0b54ea06 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:43.210393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624a027-1d5c-4de0-bce0-be1e0b54ea06 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:43.210712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624a027-1d5c-4de0-bce0-be1e0b54ea06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:43.222155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624a027-1d5c-4de0-bce0-be1e0b54ea06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:43.223636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624a027-1d5c-4de0-bce0-be1e0b54ea06 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:43.241029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-4624a027-1d5c-4de0-bce0-be1e0b54ea06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:43.241029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:43.241029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:43.241029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:43.241029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:43.241029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:43.241029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:43.241029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:43.241029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:43.241029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:43.241029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:43.241029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:43.241029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:11:43.241029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:11:43.241029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:11:43.241029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:43.242493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:11:43.243435 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:11:43.243435 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:43.243435 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:11:43.243435 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 616/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:11:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:11:43.250594 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff498f-ead3-4992-a0d7-a796a7466433 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:43.322460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:43.368064 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff498f-ead3-4992-a0d7-a796a7466433 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:43.416846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5996b6-ad61-41bd-8dbe-572846041399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:43.418016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5996b6-ad61-41bd-8dbe-572846041399 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:43.418295 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5996b6-ad61-41bd-8dbe-572846041399 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:43.418721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5996b6-ad61-41bd-8dbe-572846041399 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:43.419611 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5996b6-ad61-41bd-8dbe-572846041399 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:43.419861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5996b6-ad61-41bd-8dbe-572846041399 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:43.421395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5996b6-ad61-41bd-8dbe-572846041399 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:43.435415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 600/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:43.441463 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624a027-1d5c-4de0-bce0-be1e0b54ea06 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:43.524049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:43.566722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624a027-1d5c-4de0-bce0-be1e0b54ea06 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:43.611830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d522a4-81cb-4327-8b05-a272fa20f914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:43.612566 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d522a4-81cb-4327-8b05-a272fa20f914 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:43.612908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d522a4-81cb-4327-8b05-a272fa20f914 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:43.613250 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d522a4-81cb-4327-8b05-a272fa20f914 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:43.622036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d522a4-81cb-4327-8b05-a272fa20f914 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:43.623687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d522a4-81cb-4327-8b05-a272fa20f914 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:43.637258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-25d522a4-81cb-4327-8b05-a272fa20f914 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:11:43.637258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:43.637258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:43.637258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:43.637258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:43.637258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:43.637258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:43.637258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:43.637258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:43.637258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:43.637258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:43.637258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:43.637258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:11:43.637258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:11:43.637258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:43.638548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:43.640419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:11:43.640419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:11:43.640419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:11:43.640419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:11:43.640419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:11:43.640419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:11:43.640419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:11:43.640419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:11:43.640419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 617/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:11:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:43.645229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5996b6-ad61-41bd-8dbe-572846041399 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:43.766898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5996b6-ad61-41bd-8dbe-572846041399 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:43.791235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:43.804195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4bd70-2b87-4dec-a8e9-5a728c13777f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:43.804829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4bd70-2b87-4dec-a8e9-5a728c13777f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:43.805094 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4bd70-2b87-4dec-a8e9-5a728c13777f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:43.805369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4bd70-2b87-4dec-a8e9-5a728c13777f None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:43.806047 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4bd70-2b87-4dec-a8e9-5a728c13777f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:43.806349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4bd70-2b87-4dec-a8e9-5a728c13777f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:43.808341 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4bd70-2b87-4dec-a8e9-5a728c13777f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:43.822530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 601/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:11:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:43.828939 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d522a4-81cb-4327-8b05-a272fa20f914 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:43.937276 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d522a4-81cb-4327-8b05-a272fa20f914 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:43.967788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:43.974536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf12cf-e18b-47c6-a2e1-1ebb43e8ef34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:43.975562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf12cf-e18b-47c6-a2e1-1ebb43e8ef34 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:43.976166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf12cf-e18b-47c6-a2e1-1ebb43e8ef34 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:43.976484 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf12cf-e18b-47c6-a2e1-1ebb43e8ef34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:43.977374 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf12cf-e18b-47c6-a2e1-1ebb43e8ef34 None admin] RBAC: Adding query filter params (role.id=910821ade2904d23b75fa69e040fb82d, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=39d67f5edf4b441ebaa2bb42e74b2cd2) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:43.977711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf12cf-e18b-47c6-a2e1-1ebb43e8ef34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:43.979972 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf12cf-e18b-47c6-a2e1-1ebb43e8ef34 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:43.987961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 618/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:11:43 2022] GET /identity/v3/role_assignments?user.id=39d67f5edf4b441ebaa2bb42e74b2cd2&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=910821ade2904d23b75fa69e040fb82d => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:46.353273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7012e-8b79-4d8b-b9d2-438b196717b0 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:46.353823 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7012e-8b79-4d8b-b9d2-438b196717b0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:46.353927 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7012e-8b79-4d8b-b9d2-438b196717b0 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:46.355247 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 602/1220] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:11:46.360798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa29bd1-5254-4355-a5c0-b3faf09c316a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:46.361296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa29bd1-5254-4355-a5c0-b3faf09c316a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:46.361403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa29bd1-5254-4355-a5c0-b3faf09c316a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:46.379605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-0aa29bd1-5254-4355-a5c0-b3faf09c316a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:11:46.380812 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0aa29bd1-5254-4355-a5c0-b3faf09c316a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:11:46.449583 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 619/1221] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:11:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:11:46.724137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7012e-8b79-4d8b-b9d2-438b196717b0 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:46.839930 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7012e-8b79-4d8b-b9d2-438b196717b0 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:46.881189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509be5ea-dbab-4454-bc35-819d1a613d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:46.881989 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509be5ea-dbab-4454-bc35-819d1a613d73 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:46.882305 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509be5ea-dbab-4454-bc35-819d1a613d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:46.882560 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509be5ea-dbab-4454-bc35-819d1a613d73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:46.894991 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509be5ea-dbab-4454-bc35-819d1a613d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:11:46.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-509be5ea-dbab-4454-bc35-819d1a613d73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:11:46.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:46.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:46.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:46.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:46.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:46.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:46.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:46.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:46.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:46.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:46.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:46.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:11:46.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:11:46.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:11:46.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:11:46.902365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:11:46.902365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:11:46.902365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:11:46.902365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:11:46.902365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:46.902365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:46.902365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:46.902365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:46.902365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:46.902365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:46.902365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:46.902365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:46.902365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:46.902365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:46.902365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:46.902365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:46.903403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:46.903403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:46.903403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:46.903403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:46.903403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:11:46.903403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:11:46.903403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:11:46.903403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:11:46.903403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:11:46.903403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:11:46.903403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:11:46.903403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:11:46.903403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 603/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:11:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:46.908602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa29bd1-5254-4355-a5c0-b3faf09c316a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:47.024328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa29bd1-5254-4355-a5c0-b3faf09c316a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:47.050655 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:47.062390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8dd1b-f680-43a6-8b81-8e77d71e38ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:47.063073 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8dd1b-f680-43a6-8b81-8e77d71e38ab None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:47.063374 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8dd1b-f680-43a6-8b81-8e77d71e38ab None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:47.063750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8dd1b-f680-43a6-8b81-8e77d71e38ab None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:47.064404 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8dd1b-f680-43a6-8b81-8e77d71e38ab None admin] RBAC: Adding query filter params (name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:47.064731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8dd1b-f680-43a6-8b81-8e77d71e38ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:47.066223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8dd1b-f680-43a6-8b81-8e77d71e38ab None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:47.076519 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 620/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:11:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:11:47.083028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509be5ea-dbab-4454-bc35-819d1a613d73 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:47.128298 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:47.186402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509be5ea-dbab-4454-bc35-819d1a613d73 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:47.221617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36d72e-5438-4ec9-9be0-054e69b37993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:47.222246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36d72e-5438-4ec9-9be0-054e69b37993 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:47.222529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36d72e-5438-4ec9-9be0-054e69b37993 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:47.222803 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36d72e-5438-4ec9-9be0-054e69b37993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:47.233223 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd36d72e-5438-4ec9-9be0-054e69b37993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:47.234725 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd36d72e-5438-4ec9-9be0-054e69b37993 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:47.251694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-dd36d72e-5438-4ec9-9be0-054e69b37993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:47.251694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:47.251694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:47.251694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:47.251694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:47.251694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:47.251694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:47.251694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:47.251694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:47.251694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:47.251694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:47.251694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:47.251694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:11:47.251694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:11:47.251694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:11:47.251694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:47.252864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:11:47.254280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:11:47.254280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:47.254280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:11:47.254280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 604/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:11:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:47.258620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8dd1b-f680-43a6-8b81-8e77d71e38ab None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:47.373208 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8dd1b-f680-43a6-8b81-8e77d71e38ab None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:47.410604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a7da3-af55-432c-b9b6-d212542d941d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:47.411204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a7da3-af55-432c-b9b6-d212542d941d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:47.411481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a7da3-af55-432c-b9b6-d212542d941d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:47.411793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a7da3-af55-432c-b9b6-d212542d941d None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:47.412419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a7da3-af55-432c-b9b6-d212542d941d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:47.412768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a7da3-af55-432c-b9b6-d212542d941d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:47.414353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a7da3-af55-432c-b9b6-d212542d941d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:47.427181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 621/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:47.433135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36d72e-5438-4ec9-9be0-054e69b37993 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:47.536486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:47.544746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36d72e-5438-4ec9-9be0-054e69b37993 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:47.582101 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66e0eb-4bd3-415c-aff2-6b50ff22dd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:47.582781 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66e0eb-4bd3-415c-aff2-6b50ff22dd77 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:47.583061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66e0eb-4bd3-415c-aff2-6b50ff22dd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:47.583371 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66e0eb-4bd3-415c-aff2-6b50ff22dd77 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:47.591972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66e0eb-4bd3-415c-aff2-6b50ff22dd77 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:47.593456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66e0eb-4bd3-415c-aff2-6b50ff22dd77 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:47.611214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-cc66e0eb-4bd3-415c-aff2-6b50ff22dd77 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 12:11:47.611214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:47.611214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:47.611214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:47.611214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:47.611214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:47.611214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:47.611214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:47.611214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:47.611214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:47.611214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:47.611214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:47.611214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:11:47.611214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:11:47.611214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:11:47.611214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:47.612474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:47.613465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:47.613465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:47.613465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:11:47.613465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:11:47.613465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:11:47.613465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:11:47.613465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:11:47.613465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:11:47.613465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 12:11:47.613465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:11:47.613465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 605/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:11:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:47.618405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a7da3-af55-432c-b9b6-d212542d941d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:47.672856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:47.734566 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a7da3-af55-432c-b9b6-d212542d941d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:47.774749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e10f7-935b-401f-957c-5329c4c63f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:47.775360 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e10f7-935b-401f-957c-5329c4c63f5a None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:47.775678 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e10f7-935b-401f-957c-5329c4c63f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:47.776501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e10f7-935b-401f-957c-5329c4c63f5a None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:47.777223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e10f7-935b-401f-957c-5329c4c63f5a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:47.777527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e10f7-935b-401f-957c-5329c4c63f5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:47.778956 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e10f7-935b-401f-957c-5329c4c63f5a None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:47.796002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 622/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Oct 6 12:11:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:47.801849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66e0eb-4bd3-415c-aff2-6b50ff22dd77 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:47.915954 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66e0eb-4bd3-415c-aff2-6b50ff22dd77 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:47.952017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e4883-9592-4b38-864a-a0dfa12bc81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:47.952570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e4883-9592-4b38-864a-a0dfa12bc81f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:47.952868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e4883-9592-4b38-864a-a0dfa12bc81f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:47.953136 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e4883-9592-4b38-864a-a0dfa12bc81f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:47.969724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e4883-9592-4b38-864a-a0dfa12bc81f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:47.971676 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e4883-9592-4b38-864a-a0dfa12bc81f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:47.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-f93e4883-9592-4b38-864a-a0dfa12bc81f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:47.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:47.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:47.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:47.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:47.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:47.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:47.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:47.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:47.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:47.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:47.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:47.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:11:47.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:11:47.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:11:47.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:47.992328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:11:47.993681 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:11:47.993681 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:47.993681 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:11:47.993681 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 606/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:11:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:47.999718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e10f7-935b-401f-957c-5329c4c63f5a None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:48.103410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e10f7-935b-401f-957c-5329c4c63f5a None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:48.105773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:48.142334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c53a7b-29fb-4877-9f57-c7c7a8f1b0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:48.142949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c53a7b-29fb-4877-9f57-c7c7a8f1b0fb None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:48.143227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c53a7b-29fb-4877-9f57-c7c7a8f1b0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:48.143492 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c53a7b-29fb-4877-9f57-c7c7a8f1b0fb None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:48.144149 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c53a7b-29fb-4877-9f57-c7c7a8f1b0fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:48.144452 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c53a7b-29fb-4877-9f57-c7c7a8f1b0fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:48.145810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c53a7b-29fb-4877-9f57-c7c7a8f1b0fb None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:48.158014 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 623/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:48.164607 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e4883-9592-4b38-864a-a0dfa12bc81f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:48.174360 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:48.278168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e4883-9592-4b38-864a-a0dfa12bc81f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:48.313084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4505e-b666-458e-be79-bf1b3af3898c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:48.313706 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4505e-b666-458e-be79-bf1b3af3898c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:48.313989 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4505e-b666-458e-be79-bf1b3af3898c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:48.314266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4505e-b666-458e-be79-bf1b3af3898c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:48.321053 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4505e-b666-458e-be79-bf1b3af3898c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:48.322492 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4505e-b666-458e-be79-bf1b3af3898c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:48.333769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-ffe4505e-b666-458e-be79-bf1b3af3898c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:11:48.333769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:48.333769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:48.333769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:48.333769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:48.333769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:48.333769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:48.333769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:48.333769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:48.333769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:48.333769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:48.333769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:48.333769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:11:48.333769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:11:48.333769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:48.334869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:48.335879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:11:48.335879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:11:48.335879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:11:48.335879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:11:48.335879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:11:48.335879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:11:48.335879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:11:48.335879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:11:48.335879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 607/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:11:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:48.341058 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c53a7b-29fb-4877-9f57-c7c7a8f1b0fb None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:48.447275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c53a7b-29fb-4877-9f57-c7c7a8f1b0fb None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:48.481169 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b16e71-f5c2-48ca-a5c9-23db726f29bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:48.481775 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b16e71-f5c2-48ca-a5c9-23db726f29bd None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:48.482049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b16e71-f5c2-48ca-a5c9-23db726f29bd None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:48.482314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b16e71-f5c2-48ca-a5c9-23db726f29bd None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:48.482989 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b16e71-f5c2-48ca-a5c9-23db726f29bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:48.483293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b16e71-f5c2-48ca-a5c9-23db726f29bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:48.484824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b16e71-f5c2-48ca-a5c9-23db726f29bd None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:48.498545 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 624/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:11:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:48.504472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4505e-b666-458e-be79-bf1b3af3898c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:48.584253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:48.627742 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4505e-b666-458e-be79-bf1b3af3898c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:48.662695 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec993361-1be4-48ce-ad7e-916cb980d08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:48.663349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec993361-1be4-48ce-ad7e-916cb980d08f None admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:48.663658 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec993361-1be4-48ce-ad7e-916cb980d08f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:48.663929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec993361-1be4-48ce-ad7e-916cb980d08f None admin] PATH_INFO: `/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/39d67f5edf4b441ebaa2bb42e74b2cd2/roles/910821ade2904d23b75fa69e040fb82d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:48.678647 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec993361-1be4-48ce-ad7e-916cb980d08f None admin] RBAC: Authorizing `identity:create_grant(project_id=314d3d13ba2e4a0f93f5d3f34c50a96f, user_id=39d67f5edf4b441ebaa2bb42e74b2cd2, role_id=910821ade2904d23b75fa69e040fb82d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:48.680206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec993361-1be4-48ce-ad7e-916cb980d08f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:48.693500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 608/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:11:48 2022] PUT /identity/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/39d67f5edf4b441ebaa2bb42e74b2cd2/roles/910821ade2904d23b75fa69e040fb82d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:11:51.105279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffe820-2ad5-4a36-8299-281ef2aa356b None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:51.105850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffe820-2ad5-4a36-8299-281ef2aa356b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:51.106121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffe820-2ad5-4a36-8299-281ef2aa356b None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:51.111398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 625/1233] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:11:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:11:51.113079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7501c5-38ff-4740-abee-7b014860d290 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:51.113534 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7501c5-38ff-4740-abee-7b014860d290 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:51.113641 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7501c5-38ff-4740-abee-7b014860d290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:51.132239 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ee7501c5-38ff-4740-abee-7b014860d290 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:11:51.133433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ee7501c5-38ff-4740-abee-7b014860d290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:11:51.204617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 609/1234] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:11:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:11:51.483343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffe820-2ad5-4a36-8299-281ef2aa356b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:51.511143 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:51.599472 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffe820-2ad5-4a36-8299-281ef2aa356b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:51.637078 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e984a4b-d043-4982-9827-32b0eaba8ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:51.637713 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e984a4b-d043-4982-9827-32b0eaba8ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:51.637997 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e984a4b-d043-4982-9827-32b0eaba8ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:51.638270 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e984a4b-d043-4982-9827-32b0eaba8ef5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:51.649150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e984a4b-d043-4982-9827-32b0eaba8ef5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:11:51.653970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-6e984a4b-d043-4982-9827-32b0eaba8ef5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:11:51.653970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:51.653970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:51.653970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:51.653970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:51.653970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:51.653970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:51.653970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:51.653970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:51.653970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:51.653970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:51.653970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:51.653970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:11:51.653970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:11:51.653970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:11:51.653970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:11:51.655062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:11:51.655062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:11:51.655062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:11:51.655062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:11:51.655062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:51.655062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:51.655062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:51.655062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:51.655062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:51.655062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:51.655062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:51.655062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:51.655062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:51.655062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:51.655062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:51.655062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:51.660456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:51.660456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:51.660456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:51.660456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:51.660456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:11:51.660456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:11:51.660456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:11:51.660456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:11:51.660456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:11:51.660456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:11:51.660456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 12:11:51.660456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:11:51.660456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 626/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Oct 6 12:11:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:11:51.662205 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7501c5-38ff-4740-abee-7b014860d290 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:51.709009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:51.766143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7501c5-38ff-4740-abee-7b014860d290 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:51.803400 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8c734-8743-471b-97dc-b963582c370c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:51.804045 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8c734-8743-471b-97dc-b963582c370c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:51.804325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8c734-8743-471b-97dc-b963582c370c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:51.804600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8c734-8743-471b-97dc-b963582c370c None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:51.805283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8c734-8743-471b-97dc-b963582c370c None admin] RBAC: Adding query filter params (name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:51.805574 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8c734-8743-471b-97dc-b963582c370c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:51.807468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8c734-8743-471b-97dc-b963582c370c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:51.815610 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 610/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Oct 6 12:11:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:11:51.821694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e984a4b-d043-4982-9827-32b0eaba8ef5 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:51.908253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:51.926674 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e984a4b-d043-4982-9827-32b0eaba8ef5 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:51.973109 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca8823-aaf6-4a71-a170-c8d39611b535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:51.973814 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca8823-aaf6-4a71-a170-c8d39611b535 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:51.974134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca8823-aaf6-4a71-a170-c8d39611b535 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:51.974408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca8823-aaf6-4a71-a170-c8d39611b535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:51.989540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca8823-aaf6-4a71-a170-c8d39611b535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:51.991016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca8823-aaf6-4a71-a170-c8d39611b535 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:52.007656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-78ca8823-aaf6-4a71-a170-c8d39611b535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:52.007656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:52.007656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:52.007656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:52.007656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:52.007656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:52.007656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:52.007656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:52.007656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:52.007656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:52.007656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:52.007656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:52.007656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:11:52.007656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:11:52.007656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:11:52.007656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:52.008700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:11:52.009648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:11:52.009648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:52.009648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:11:52.009648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 627/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:11:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:52.016845 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8c734-8743-471b-97dc-b963582c370c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:52.130193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8c734-8743-471b-97dc-b963582c370c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:52.139939 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:52.167044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56595c-44f6-4aea-9e31-89922e8bb09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:52.167714 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56595c-44f6-4aea-9e31-89922e8bb09d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:52.167998 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56595c-44f6-4aea-9e31-89922e8bb09d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:52.168271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56595c-44f6-4aea-9e31-89922e8bb09d None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:52.168915 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56595c-44f6-4aea-9e31-89922e8bb09d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:52.169215 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56595c-44f6-4aea-9e31-89922e8bb09d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:52.170582 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56595c-44f6-4aea-9e31-89922e8bb09d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:52.183666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 611/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:52.189361 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca8823-aaf6-4a71-a170-c8d39611b535 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:52.300405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca8823-aaf6-4a71-a170-c8d39611b535 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:52.336855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67366283-43e8-49aa-b8c0-12212b938caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:52.337645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67366283-43e8-49aa-b8c0-12212b938caf None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:52.338114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67366283-43e8-49aa-b8c0-12212b938caf None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:52.338457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67366283-43e8-49aa-b8c0-12212b938caf None admin] PATH_INFO: `/v3/users/placement` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:52.341860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:52.350076 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67366283-43e8-49aa-b8c0-12212b938caf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:52.352110 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67366283-43e8-49aa-b8c0-12212b938caf None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:52.366057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-67366283-43e8-49aa-b8c0-12212b938caf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 12:11:52.366057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:52.366057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:52.366057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:52.366057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:52.366057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:52.366057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:52.366057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:52.366057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:52.366057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:52.366057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:52.366057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:52.366057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:11:52.366057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:11:52.366057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:11:52.366057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:52.367162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:52.368222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:52.368222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:52.368222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:11:52.368222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:11:52.368222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:11:52.368222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:11:52.368222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:11:52.368222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:11:52.368222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 12:11:52.368222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:11:52.368222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 628/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:11:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:11:52.374067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56595c-44f6-4aea-9e31-89922e8bb09d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:52.490121 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56595c-44f6-4aea-9e31-89922e8bb09d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:52.527354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860908c4-86c0-4fda-bcb9-f0fbbfcf51f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:52.528013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860908c4-86c0-4fda-bcb9-f0fbbfcf51f4 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:52.528294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860908c4-86c0-4fda-bcb9-f0fbbfcf51f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:52.528567 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860908c4-86c0-4fda-bcb9-f0fbbfcf51f4 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:52.529298 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860908c4-86c0-4fda-bcb9-f0fbbfcf51f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:52.529602 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860908c4-86c0-4fda-bcb9-f0fbbfcf51f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:52.531082 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860908c4-86c0-4fda-bcb9-f0fbbfcf51f4 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:52.550103 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 612/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Oct 6 12:11:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:52.556538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67366283-43e8-49aa-b8c0-12212b938caf None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:52.668217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67366283-43e8-49aa-b8c0-12212b938caf None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:52.703856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395f8a2-6f94-489d-8e6c-9e230f150be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:52.704460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395f8a2-6f94-489d-8e6c-9e230f150be7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:52.704740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395f8a2-6f94-489d-8e6c-9e230f150be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:52.705010 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395f8a2-6f94-489d-8e6c-9e230f150be7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:52.770704 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:52.779020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395f8a2-6f94-489d-8e6c-9e230f150be7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:52.780551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395f8a2-6f94-489d-8e6c-9e230f150be7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:52.896943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 629/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:11:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:11:52.898198 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860908c4-86c0-4fda-bcb9-f0fbbfcf51f4 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:52.927788 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:53.015464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860908c4-86c0-4fda-bcb9-f0fbbfcf51f4 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:53.055343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ac3cb-b096-42e4-9dea-ed4690d68546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:53.056094 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ac3cb-b096-42e4-9dea-ed4690d68546 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:53.056385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ac3cb-b096-42e4-9dea-ed4690d68546 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:53.056706 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ac3cb-b096-42e4-9dea-ed4690d68546 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:53.067529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ac3cb-b096-42e4-9dea-ed4690d68546 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:53.069113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ac3cb-b096-42e4-9dea-ed4690d68546 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:53.085349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-834ac3cb-b096-42e4-9dea-ed4690d68546 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:53.085349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:53.085349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:53.085349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:53.085349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:53.085349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:53.085349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:53.085349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:53.085349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:53.085349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:53.085349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:53.085349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:53.085349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:11:53.085349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:11:53.085349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:11:53.085349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:53.086362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:11:53.087288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:11:53.087288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:53.087288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:11:53.087288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 613/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:11:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:11:53.092820 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395f8a2-6f94-489d-8e6c-9e230f150be7 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:53.203483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:53.209676 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395f8a2-6f94-489d-8e6c-9e230f150be7 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:53.247396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae34a0-68bd-4369-ae7f-58abaa28a123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:53.248059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae34a0-68bd-4369-ae7f-58abaa28a123 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:53.248343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae34a0-68bd-4369-ae7f-58abaa28a123 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:53.248614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae34a0-68bd-4369-ae7f-58abaa28a123 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:53.249264 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae34a0-68bd-4369-ae7f-58abaa28a123 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:53.249571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae34a0-68bd-4369-ae7f-58abaa28a123 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:53.250934 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae34a0-68bd-4369-ae7f-58abaa28a123 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:53.262760 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 630/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:53.268810 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ac3cb-b096-42e4-9dea-ed4690d68546 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:53.360685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:53.385433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ac3cb-b096-42e4-9dea-ed4690d68546 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:53.428617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2cac8-8707-4ac5-964b-19ffd789771c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:53.429248 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2cac8-8707-4ac5-964b-19ffd789771c None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:53.429532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2cac8-8707-4ac5-964b-19ffd789771c None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:53.429796 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2cac8-8707-4ac5-964b-19ffd789771c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:53.438055 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2cac8-8707-4ac5-964b-19ffd789771c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:53.439774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2cac8-8707-4ac5-964b-19ffd789771c None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:53.453388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-05b2cac8-8707-4ac5-964b-19ffd789771c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:11:53.453388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:53.453388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:53.453388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:53.453388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:53.453388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:53.453388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:53.453388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:53.453388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:53.453388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:53.453388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:53.453388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:53.453388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:11:53.453388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:11:53.453388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:53.455035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:53.456234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:11:53.456234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:11:53.456234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:11:53.456234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:11:53.456234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:11:53.456234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:11:53.456234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:11:53.456234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:11:53.456234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 614/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:11:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:53.460979 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae34a0-68bd-4369-ae7f-58abaa28a123 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:53.573125 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae34a0-68bd-4369-ae7f-58abaa28a123 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:53.612031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b681fe7-319a-4d1a-a8fd-4bf774892859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:53.612642 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b681fe7-319a-4d1a-a8fd-4bf774892859 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:53.612930 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b681fe7-319a-4d1a-a8fd-4bf774892859 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:53.613205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b681fe7-319a-4d1a-a8fd-4bf774892859 None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:53.613893 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b681fe7-319a-4d1a-a8fd-4bf774892859 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:53.614203 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b681fe7-319a-4d1a-a8fd-4bf774892859 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:53.615795 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b681fe7-319a-4d1a-a8fd-4bf774892859 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:53.628771 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 631/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:11:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:53.635778 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2cac8-8707-4ac5-964b-19ffd789771c None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:53.752184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2cac8-8707-4ac5-964b-19ffd789771c None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:53.790496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c79f6e-c05e-4fd9-8b99-f3f3538361c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:53.791682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c79f6e-c05e-4fd9-8b99-f3f3538361c8 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:53.791931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c79f6e-c05e-4fd9-8b99-f3f3538361c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:53.792198 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c79f6e-c05e-4fd9-8b99-f3f3538361c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:53.792912 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c79f6e-c05e-4fd9-8b99-f3f3538361c8 None admin] RBAC: Adding query filter params (role.id=910821ade2904d23b75fa69e040fb82d, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=39d67f5edf4b441ebaa2bb42e74b2cd2) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:53.793214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c79f6e-c05e-4fd9-8b99-f3f3538361c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:53.794755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c79f6e-c05e-4fd9-8b99-f3f3538361c8 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:53.800964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 615/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:11:53 2022] GET /identity/v3/role_assignments?user.id=39d67f5edf4b441ebaa2bb42e74b2cd2&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=910821ade2904d23b75fa69e040fb82d => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:56.234998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af7433-9544-4d34-9a90-23ee774be16a None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:56.235706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af7433-9544-4d34-9a90-23ee774be16a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:56.235706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af7433-9544-4d34-9a90-23ee774be16a None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:56.236897 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 632/1247] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:11:56.242203 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e88c8-b8d0-4dae-8836-3fa27d378ee7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:56.242662 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e88c8-b8d0-4dae-8836-3fa27d378ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:56.242764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e88c8-b8d0-4dae-8836-3fa27d378ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:56.249818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:56.263249 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-706e88c8-b8d0-4dae-8836-3fa27d378ee7 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:11:56.264477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-706e88c8-b8d0-4dae-8836-3fa27d378ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:11:56.334679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 616/1248] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:11:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:11:56.601127 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af7433-9544-4d34-9a90-23ee774be16a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:56.620651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:56.709743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af7433-9544-4d34-9a90-23ee774be16a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:56.747720 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1d088-fbb1-4e82-a945-d478b5c35b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:56.748350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1d088-fbb1-4e82-a945-d478b5c35b14 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:56.748632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1d088-fbb1-4e82-a945-d478b5c35b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:56.748908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1d088-fbb1-4e82-a945-d478b5c35b14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:56.760193 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1d088-fbb1-4e82-a945-d478b5c35b14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:11:56.765256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-30f1d088-fbb1-4e82-a945-d478b5c35b14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:11:56.765256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:56.765256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:56.765256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:56.765256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:56.765256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:56.765256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:56.765256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:56.765256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:56.765256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:56.765256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:56.765256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:56.765256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:11:56.765256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:11:56.765256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:11:56.765256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:11:56.766343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:11:56.766343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:11:56.766343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:11:56.766343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:11:56.766343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:56.766343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:56.766343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:56.766343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:56.766343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:56.766343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:56.766343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:56.766343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:56.766343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:56.766343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:56.766343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:56.766343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:56.767927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:56.767927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:56.767927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:56.767927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:56.767927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:11:56.767927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:11:56.767927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:11:56.767927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:11:56.767927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:11:56.767927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:11:56.767927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:11:56.767927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:11:56.767927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 633/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:11:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:56.772972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e88c8-b8d0-4dae-8836-3fa27d378ee7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:56.874220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e88c8-b8d0-4dae-8836-3fa27d378ee7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:56.904852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:56.910677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1eb785-a42c-4bb4-a62f-bd408cf393bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:56.911332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1eb785-a42c-4bb4-a62f-bd408cf393bc None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:56.911641 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1eb785-a42c-4bb4-a62f-bd408cf393bc None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:56.911916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1eb785-a42c-4bb4-a62f-bd408cf393bc None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:56.912583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1eb785-a42c-4bb4-a62f-bd408cf393bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:56.912885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1eb785-a42c-4bb4-a62f-bd408cf393bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:56.914299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1eb785-a42c-4bb4-a62f-bd408cf393bc None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:56.924080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 617/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:11:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:56.930049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1d088-fbb1-4e82-a945-d478b5c35b14 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:57.029121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:57.055027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1d088-fbb1-4e82-a945-d478b5c35b14 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:57.090764 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51006b46-0f56-44a3-b7d8-4e260481abf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:57.091409 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51006b46-0f56-44a3-b7d8-4e260481abf0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:57.091719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51006b46-0f56-44a3-b7d8-4e260481abf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:57.091992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51006b46-0f56-44a3-b7d8-4e260481abf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:57.103099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51006b46-0f56-44a3-b7d8-4e260481abf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:57.104817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51006b46-0f56-44a3-b7d8-4e260481abf0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:57.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-51006b46-0f56-44a3-b7d8-4e260481abf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:57.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:57.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:57.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:57.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:57.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:57.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:57.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:57.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:57.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:57.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:57.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:57.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:11:57.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:11:57.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:11:57.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:57.123408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:11:57.124552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:11:57.124552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:57.124552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:11:57.124552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 634/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:11:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:57.129137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1eb785-a42c-4bb4-a62f-bd408cf393bc None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:57.242285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1eb785-a42c-4bb4-a62f-bd408cf393bc None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:57.279778 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e979147-2b0d-488c-a9bc-4cf89513940f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:57.280400 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e979147-2b0d-488c-a9bc-4cf89513940f None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:57.280681 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e979147-2b0d-488c-a9bc-4cf89513940f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:57.280953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e979147-2b0d-488c-a9bc-4cf89513940f None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:57.281599 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e979147-2b0d-488c-a9bc-4cf89513940f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:57.281903 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e979147-2b0d-488c-a9bc-4cf89513940f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:57.283307 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e979147-2b0d-488c-a9bc-4cf89513940f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:57.296271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 618/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:57.302068 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51006b46-0f56-44a3-b7d8-4e260481abf0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:57.407798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51006b46-0f56-44a3-b7d8-4e260481abf0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:57.434566 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:57.442717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b145a3-3b9d-4904-bcb0-445a5de2d5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:57.443361 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b145a3-3b9d-4904-bcb0-445a5de2d5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:57.443688 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b145a3-3b9d-4904-bcb0-445a5de2d5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:57.443966 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b145a3-3b9d-4904-bcb0-445a5de2d5c7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:57.452992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b145a3-3b9d-4904-bcb0-445a5de2d5c7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:57.454467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b145a3-3b9d-4904-bcb0-445a5de2d5c7 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:57.467473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-70b145a3-3b9d-4904-bcb0-445a5de2d5c7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 12:11:57.467473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:57.467473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:57.467473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:57.467473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:57.467473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:57.467473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:57.467473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:57.467473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:57.467473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:57.467473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:57.467473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:57.467473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:11:57.467473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:11:57.467473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:11:57.467473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:57.468768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:57.469765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:57.469765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:57.469765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:11:57.469765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:11:57.469765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:11:57.469765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:11:57.469765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:11:57.469765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:11:57.469765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 12:11:57.469765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:11:57.469765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 635/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:11:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:57.474649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e979147-2b0d-488c-a9bc-4cf89513940f None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:57.533118 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:57.578240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e979147-2b0d-488c-a9bc-4cf89513940f None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:57.612306 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371bd1d-dca1-4264-9d75-15dab7465030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:57.612912 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371bd1d-dca1-4264-9d75-15dab7465030 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:57.613199 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371bd1d-dca1-4264-9d75-15dab7465030 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:57.613475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371bd1d-dca1-4264-9d75-15dab7465030 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:57.614226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371bd1d-dca1-4264-9d75-15dab7465030 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:57.614543 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371bd1d-dca1-4264-9d75-15dab7465030 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:57.616097 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371bd1d-dca1-4264-9d75-15dab7465030 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:57.633065 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 619/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Oct 6 12:11:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:57.639341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b145a3-3b9d-4904-bcb0-445a5de2d5c7 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:57.740766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b145a3-3b9d-4904-bcb0-445a5de2d5c7 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:57.774828 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7221e7f-8926-4d97-b2d3-44eb12bb4b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:57.775458 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7221e7f-8926-4d97-b2d3-44eb12bb4b97 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:57.775772 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7221e7f-8926-4d97-b2d3-44eb12bb4b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:57.776043 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7221e7f-8926-4d97-b2d3-44eb12bb4b97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:57.832789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:57.845164 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7221e7f-8926-4d97-b2d3-44eb12bb4b97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:57.846610 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7221e7f-8926-4d97-b2d3-44eb12bb4b97 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:57.941067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 636/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:11:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:11:57.946878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371bd1d-dca1-4264-9d75-15dab7465030 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:58.052253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371bd1d-dca1-4264-9d75-15dab7465030 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:58.065348 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:58.085730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48ba25-98f2-40c0-9636-75fcd0f2b6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:58.086351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48ba25-98f2-40c0-9636-75fcd0f2b6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:58.086632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48ba25-98f2-40c0-9636-75fcd0f2b6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:58.086907 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48ba25-98f2-40c0-9636-75fcd0f2b6f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:58.096576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48ba25-98f2-40c0-9636-75fcd0f2b6f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:58.098006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48ba25-98f2-40c0-9636-75fcd0f2b6f7 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:58.112523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-fa48ba25-98f2-40c0-9636-75fcd0f2b6f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:58.112523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:58.112523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:58.112523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:58.112523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:58.112523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:58.112523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:58.112523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:58.112523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:58.112523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:58.112523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:58.112523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:58.112523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:11:58.112523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:11:58.112523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:11:58.112523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:58.113666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:11:58.114775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:11:58.114775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:11:58.114775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:11:58.114775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 620/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:11:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:58.119949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7221e7f-8926-4d97-b2d3-44eb12bb4b97 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:58.203720 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:58.215327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7221e7f-8926-4d97-b2d3-44eb12bb4b97 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:58.249034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de12e27-a5fe-4891-aaf8-d5fdf8b255d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:58.249634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de12e27-a5fe-4891-aaf8-d5fdf8b255d0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:58.249914 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de12e27-a5fe-4891-aaf8-d5fdf8b255d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:58.250185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de12e27-a5fe-4891-aaf8-d5fdf8b255d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:58.250816 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de12e27-a5fe-4891-aaf8-d5fdf8b255d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:58.251117 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de12e27-a5fe-4891-aaf8-d5fdf8b255d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:58.252513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de12e27-a5fe-4891-aaf8-d5fdf8b255d0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:58.263554 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 637/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:11:58.269246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48ba25-98f2-40c0-9636-75fcd0f2b6f7 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:58.364730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48ba25-98f2-40c0-9636-75fcd0f2b6f7 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:58.396582 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7ccfb-176a-4461-9c2b-8c53ddb91f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:58.397186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7ccfb-176a-4461-9c2b-8c53ddb91f6e None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:58.397465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7ccfb-176a-4461-9c2b-8c53ddb91f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:58.397736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7ccfb-176a-4461-9c2b-8c53ddb91f6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:58.404905 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7ccfb-176a-4461-9c2b-8c53ddb91f6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:58.406332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7ccfb-176a-4461-9c2b-8c53ddb91f6e None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:58.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-6ee7ccfb-176a-4461-9c2b-8c53ddb91f6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:11:58.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:11:58.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:11:58.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:11:58.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:11:58.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:11:58.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:11:58.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:11:58.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:11:58.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:11:58.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:11:58.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:11:58.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:11:58.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:11:58.417735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:11:58.418840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:11:58.419993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:11:58.419993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:11:58.419993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:11:58.419993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:11:58.419993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:11:58.419993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:11:58.419993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:11:58.419993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:11:58.419993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 621/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:11:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:11:58.424555 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de12e27-a5fe-4891-aaf8-d5fdf8b255d0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:58.520665 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de12e27-a5fe-4891-aaf8-d5fdf8b255d0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:58.553427 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4493131-42ad-4d9c-8680-d8c93762c13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:58.554016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4493131-42ad-4d9c-8680-d8c93762c13c None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:58.554293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4493131-42ad-4d9c-8680-d8c93762c13c None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:58.554561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4493131-42ad-4d9c-8680-d8c93762c13c None admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:58.555236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4493131-42ad-4d9c-8680-d8c93762c13c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:58.555562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4493131-42ad-4d9c-8680-d8c93762c13c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:58.557012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4493131-42ad-4d9c-8680-d8c93762c13c None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:58.568422 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 638/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:11:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:11:58.574512 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7ccfb-176a-4461-9c2b-8c53ddb91f6e None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:11:58.599533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:11:58.671421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7ccfb-176a-4461-9c2b-8c53ddb91f6e None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:11:58.702664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f219e-94ec-44f6-8c8e-b4cfaf7f853a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:11:58.703246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f219e-94ec-44f6-8c8e-b4cfaf7f853a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:11:58.703545 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f219e-94ec-44f6-8c8e-b4cfaf7f853a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:11:58.703823 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f219e-94ec-44f6-8c8e-b4cfaf7f853a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:11:58.704527 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f219e-94ec-44f6-8c8e-b4cfaf7f853a None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=39d67f5edf4b441ebaa2bb42e74b2cd2) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:11:58.704832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f219e-94ec-44f6-8c8e-b4cfaf7f853a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:11:58.706239 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f219e-94ec-44f6-8c8e-b4cfaf7f853a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:11:58.711338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 622/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:11:58 2022] GET /identity/v3/role_assignments?user.id=39d67f5edf4b441ebaa2bb42e74b2cd2&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:12:00.817304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e20ec-2907-4716-9e37-e4b64970a2bc None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:00.817842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e20ec-2907-4716-9e37-e4b64970a2bc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:00.817842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e20ec-2907-4716-9e37-e4b64970a2bc None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:00.819160 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 639/1261] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 12:12:00.824836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e508034-9580-4e00-8778-280d89edd7ea None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:00.825238 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e508034-9580-4e00-8778-280d89edd7ea None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:00.825376 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e508034-9580-4e00-8778-280d89edd7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:00.842941 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9e508034-9580-4e00-8778-280d89edd7ea None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:12:00.844132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9e508034-9580-4e00-8778-280d89edd7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:12:00.907568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 623/1262] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:12:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:12:01.175521 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e20ec-2907-4716-9e37-e4b64970a2bc None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:01.186908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:01.273744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e20ec-2907-4716-9e37-e4b64970a2bc None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:01.306443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cc85c-d071-452a-bb8d-f6be7eb4edbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:01.307087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cc85c-d071-452a-bb8d-f6be7eb4edbf None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:01.307369 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cc85c-d071-452a-bb8d-f6be7eb4edbf None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:01.307670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cc85c-d071-452a-bb8d-f6be7eb4edbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:01.317321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917cc85c-d071-452a-bb8d-f6be7eb4edbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:12:01.322118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-917cc85c-d071-452a-bb8d-f6be7eb4edbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:12:01.322118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:12:01.322118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:12:01.322118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:12:01.322118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:12:01.322118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:12:01.322118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:12:01.322118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:12:01.322118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:12:01.322118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:12:01.322118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:12:01.322118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:12:01.322118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:12:01.322118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:12:01.322118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:12:01.322118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:12:01.323235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:12:01.323235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:12:01.323235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:12:01.323235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:12:01.323235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:12:01.323235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:12:01.323235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:12:01.323235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:12:01.323235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:12:01.323235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:12:01.323235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:12:01.323235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:12:01.323235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:12:01.323235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:12:01.323235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:12:01.323235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:12:01.324213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:12:01.324213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:12:01.324213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:12:01.324213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:12:01.324213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:12:01.324213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:12:01.324213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:12:01.324213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:12:01.324213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:12:01.324213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:12:01.324213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:12:01.324213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:12:01.324213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 640/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:12:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:12:01.329108 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e508034-9580-4e00-8778-280d89edd7ea None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:01.346559 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:01.424804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e508034-9580-4e00-8778-280d89edd7ea None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:01.456753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15c033-9b01-4551-9983-9c8f69006c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:01.457293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15c033-9b01-4551-9983-9c8f69006c90 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:01.457571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15c033-9b01-4551-9983-9c8f69006c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:01.457847 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15c033-9b01-4551-9983-9c8f69006c90 None admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:01.458500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15c033-9b01-4551-9983-9c8f69006c90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:12:01.458804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15c033-9b01-4551-9983-9c8f69006c90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:01.460193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15c033-9b01-4551-9983-9c8f69006c90 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:01.468448 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 624/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:12:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:12:01.474196 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cc85c-d071-452a-bb8d-f6be7eb4edbf None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:01.538388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:01.570153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cc85c-d071-452a-bb8d-f6be7eb4edbf None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:01.602181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0335eb2-fca4-4eea-8d8f-3ede32177473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:01.602836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0335eb2-fca4-4eea-8d8f-3ede32177473 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:01.603637 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0335eb2-fca4-4eea-8d8f-3ede32177473 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:01.603908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0335eb2-fca4-4eea-8d8f-3ede32177473 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:01.613617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0335eb2-fca4-4eea-8d8f-3ede32177473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:01.615717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0335eb2-fca4-4eea-8d8f-3ede32177473 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:01.630299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-c0335eb2-fca4-4eea-8d8f-3ede32177473 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:12:01.630299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:12:01.630299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:12:01.630299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:12:01.630299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:12:01.630299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:12:01.630299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:12:01.630299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:12:01.630299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:12:01.630299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:12:01.630299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:12:01.630299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:12:01.630299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:12:01.630299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:12:01.630299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:12:01.630299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:12:01.631441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:12:01.632577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:12:01.632577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:12:01.632577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:12:01.632577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 641/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:12:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 12:12:01.637171 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15c033-9b01-4551-9983-9c8f69006c90 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:01.716496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:01.735838 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15c033-9b01-4551-9983-9c8f69006c90 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:01.767512 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea92bb5-00e5-4dd7-ba68-18867a462583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:01.768126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea92bb5-00e5-4dd7-ba68-18867a462583 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:01.768405 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea92bb5-00e5-4dd7-ba68-18867a462583 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:01.768674 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea92bb5-00e5-4dd7-ba68-18867a462583 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:01.769300 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea92bb5-00e5-4dd7-ba68-18867a462583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:12:01.769598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea92bb5-00e5-4dd7-ba68-18867a462583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:01.770950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea92bb5-00e5-4dd7-ba68-18867a462583 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:01.782893 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 625/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:12:01.788476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0335eb2-fca4-4eea-8d8f-3ede32177473 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:01.883457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0335eb2-fca4-4eea-8d8f-3ede32177473 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:01.903072 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:01.916621 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d9378-20a1-470e-9610-458dde5a18c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:01.917225 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d9378-20a1-470e-9610-458dde5a18c6 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:01.917502 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d9378-20a1-470e-9610-458dde5a18c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:01.917769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d9378-20a1-470e-9610-458dde5a18c6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:01.924831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d9378-20a1-470e-9610-458dde5a18c6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:01.926244 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d9378-20a1-470e-9610-458dde5a18c6 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:01.936902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-a26d9378-20a1-470e-9610-458dde5a18c6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 12:12:01.936902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:12:01.936902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:12:01.936902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:12:01.936902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:12:01.936902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:12:01.936902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:12:01.936902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:12:01.936902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:12:01.936902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:12:01.936902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:12:01.936902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:12:01.936902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:12:01.936902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:12:01.936902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:12:01.936902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:12:01.938028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:12:01.939243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:12:01.939243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:12:01.939243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:12:01.939243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:12:01.939243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:12:01.939243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:12:01.939243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:12:01.939243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:12:01.939243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 12:12:01.939243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:12:01.939243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 642/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:12:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:12:01.943751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea92bb5-00e5-4dd7-ba68-18867a462583 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:02.044578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea92bb5-00e5-4dd7-ba68-18867a462583 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:02.076866 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32934ce6-1faf-4fc1-baa0-474eb01023a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:02.077463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32934ce6-1faf-4fc1-baa0-474eb01023a0 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:02.077744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32934ce6-1faf-4fc1-baa0-474eb01023a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:02.078015 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32934ce6-1faf-4fc1-baa0-474eb01023a0 None admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:02.078730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32934ce6-1faf-4fc1-baa0-474eb01023a0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:12:02.079032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32934ce6-1faf-4fc1-baa0-474eb01023a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:02.080476 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32934ce6-1faf-4fc1-baa0-474eb01023a0 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:02.089231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:02.097584 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 626/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Oct 6 12:12:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:12:02.103464 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d9378-20a1-470e-9610-458dde5a18c6 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:02.198371 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d9378-20a1-470e-9610-458dde5a18c6 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:02.230547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65116358-48eb-4405-988f-b19e2a75751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:02.231405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65116358-48eb-4405-988f-b19e2a75751e None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:02.231886 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65116358-48eb-4405-988f-b19e2a75751e None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:02.232275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65116358-48eb-4405-988f-b19e2a75751e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:02.243109 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65116358-48eb-4405-988f-b19e2a75751e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:02.245086 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65116358-48eb-4405-988f-b19e2a75751e None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:02.259142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-65116358-48eb-4405-988f-b19e2a75751e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:12:02.259142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:12:02.259142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:12:02.259142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:12:02.259142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:12:02.259142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:12:02.259142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:12:02.259142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:12:02.259142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:12:02.259142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:12:02.259142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:12:02.259142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:12:02.259142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:12:02.259142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:12:02.259142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:12:02.259142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:12:02.260300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:12:02.261580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:12:02.261580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:12:02.261580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:12:02.261580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 643/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:12:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:12:02.266527 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32934ce6-1faf-4fc1-baa0-474eb01023a0 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:02.361584 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32934ce6-1faf-4fc1-baa0-474eb01023a0 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:02.394628 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1f88b-aeca-4420-bd25-2d2cbe54a707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:02.395220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1f88b-aeca-4420-bd25-2d2cbe54a707 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:02.395527 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1f88b-aeca-4420-bd25-2d2cbe54a707 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:02.395809 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1f88b-aeca-4420-bd25-2d2cbe54a707 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:02.396441 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1f88b-aeca-4420-bd25-2d2cbe54a707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:12:02.396744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1f88b-aeca-4420-bd25-2d2cbe54a707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:02.398353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1f88b-aeca-4420-bd25-2d2cbe54a707 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:02.408974 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 627/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:12:02.414454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65116358-48eb-4405-988f-b19e2a75751e None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:02.445428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:02.513027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65116358-48eb-4405-988f-b19e2a75751e None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:02.545507 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b8770-ad0f-462f-a308-5db7c005b429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:02.546218 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b8770-ad0f-462f-a308-5db7c005b429 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:02.546500 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b8770-ad0f-462f-a308-5db7c005b429 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:02.546773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b8770-ad0f-462f-a308-5db7c005b429 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:02.553214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b8770-ad0f-462f-a308-5db7c005b429 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:02.554639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b8770-ad0f-462f-a308-5db7c005b429 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:02.566230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-3b3b8770-ad0f-462f-a308-5db7c005b429 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:12:02.566230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:12:02.566230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:12:02.566230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:12:02.566230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:12:02.566230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:12:02.566230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:12:02.566230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:12:02.566230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:12:02.566230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:12:02.566230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:12:02.566230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:12:02.566230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:12:02.566230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:12:02.566230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:12:02.567300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:12:02.568470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:12:02.568470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:12:02.568470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:12:02.568470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:12:02.568470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:12:02.568470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:12:02.568470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:12:02.568470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:12:02.568470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 644/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:12:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:12:02.573128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1f88b-aeca-4420-bd25-2d2cbe54a707 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:02.629389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:02.670800 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1f88b-aeca-4420-bd25-2d2cbe54a707 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:02.702583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde3833-ccd1-4824-9ef6-728330689030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:02.703180 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde3833-ccd1-4824-9ef6-728330689030 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:02.703464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde3833-ccd1-4824-9ef6-728330689030 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:02.703770 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde3833-ccd1-4824-9ef6-728330689030 None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:02.704453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde3833-ccd1-4824-9ef6-728330689030 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:12:02.704759 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde3833-ccd1-4824-9ef6-728330689030 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:02.706184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde3833-ccd1-4824-9ef6-728330689030 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:02.718180 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 628/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:12:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:12:02.724035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b8770-ad0f-462f-a308-5db7c005b429 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:02.812447 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:02.821177 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b8770-ad0f-462f-a308-5db7c005b429 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:02.853818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a6ecf-944f-4a5c-80da-c0673d12ba89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:02.854455 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a6ecf-944f-4a5c-80da-c0673d12ba89 None admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:02.854735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a6ecf-944f-4a5c-80da-c0673d12ba89 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:02.855009 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a6ecf-944f-4a5c-80da-c0673d12ba89 None admin] PATH_INFO: `/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/39d67f5edf4b441ebaa2bb42e74b2cd2/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:02.856909 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a6ecf-944f-4a5c-80da-c0673d12ba89 None admin] RBAC: Authorizing `identity:create_grant(project_id=314d3d13ba2e4a0f93f5d3f34c50a96f, user_id=39d67f5edf4b441ebaa2bb42e74b2cd2, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:02.858328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a6ecf-944f-4a5c-80da-c0673d12ba89 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:02.870160 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 645/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Oct 6 12:12:02 2022] PUT /identity/v3/projects/314d3d13ba2e4a0f93f5d3f34c50a96f/users/39d67f5edf4b441ebaa2bb42e74b2cd2/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:12:04.962743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326a62e-0651-4f62-85e1-2cdfa1416f98 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:04.963404 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326a62e-0651-4f62-85e1-2cdfa1416f98 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:04.963404 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326a62e-0651-4f62-85e1-2cdfa1416f98 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:04.964630 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 629/1274] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:12:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:12:04.970331 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f18252-d0d3-4733-9255-dd4a77491696 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:04.970803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f18252-d0d3-4733-9255-dd4a77491696 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:04.970955 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f18252-d0d3-4733-9255-dd4a77491696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:04.993236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-55f18252-d0d3-4733-9255-dd4a77491696 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:12:04.994736 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-55f18252-d0d3-4733-9255-dd4a77491696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:12:05.067123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 646/1275] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:12:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:12:05.333777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326a62e-0651-4f62-85e1-2cdfa1416f98 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:05.431361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326a62e-0651-4f62-85e1-2cdfa1416f98 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:05.449035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:05.465885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ac41f-e947-43f4-a128-26e081aee1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:05.466499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ac41f-e947-43f4-a128-26e081aee1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:05.466778 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ac41f-e947-43f4-a128-26e081aee1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:05.467053 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ac41f-e947-43f4-a128-26e081aee1f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:05.476841 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ac41f-e947-43f4-a128-26e081aee1f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 12:12:05.481846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-df8ac41f-e947-43f4-a128-26e081aee1f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:12:05.481846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:12:05.481846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:12:05.481846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:12:05.481846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:12:05.481846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:12:05.481846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:12:05.481846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:12:05.481846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:12:05.481846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:12:05.481846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:12:05.481846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:12:05.481846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 12:12:05.481846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 12:12:05.481846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 12:12:05.481846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 12:12:05.483026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 12:12:05.483026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 12:12:05.483026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 12:12:05.483026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 12:12:05.483026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:12:05.483026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:12:05.483026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:12:05.483026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:12:05.483026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:12:05.483026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:12:05.483026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:12:05.483026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:12:05.483026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:12:05.483026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:12:05.483026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:12:05.483026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:12:05.484181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:12:05.484181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:12:05.484181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:12:05.484181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:12:05.484181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 12:12:05.484181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 12:12:05.484181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 12:12:05.484181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 12:12:05.484181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 12:12:05.484181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 12:12:05.484181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 12:12:05.484181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:12:05.484181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 630/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Oct 6 12:12:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:12:05.489264 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f18252-d0d3-4733-9255-dd4a77491696 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:05.572744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:05.592222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f18252-d0d3-4733-9255-dd4a77491696 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:05.627181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4682d9-4bc1-412e-85b0-dceec327a049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:05.627812 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4682d9-4bc1-412e-85b0-dceec327a049 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:05.628094 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4682d9-4bc1-412e-85b0-dceec327a049 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:05.628365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4682d9-4bc1-412e-85b0-dceec327a049 None admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:05.629020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4682d9-4bc1-412e-85b0-dceec327a049 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:12:05.629322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4682d9-4bc1-412e-85b0-dceec327a049 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:05.630678 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4682d9-4bc1-412e-85b0-dceec327a049 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:05.640334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 647/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Oct 6 12:12:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:12:05.646139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ac41f-e947-43f4-a128-26e081aee1f6 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:05.740752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ac41f-e947-43f4-a128-26e081aee1f6 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:05.773958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfef727-2de3-47fa-ad73-61f8a3fba9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:05.774609 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfef727-2de3-47fa-ad73-61f8a3fba9be None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:05.774884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfef727-2de3-47fa-ad73-61f8a3fba9be None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:05.775151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfef727-2de3-47fa-ad73-61f8a3fba9be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:05.784765 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfef727-2de3-47fa-ad73-61f8a3fba9be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:05.786267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfef727-2de3-47fa-ad73-61f8a3fba9be None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:05.800639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-6cfef727-2de3-47fa-ad73-61f8a3fba9be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:12:05.800639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:12:05.800639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:12:05.800639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:12:05.800639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:12:05.800639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:12:05.800639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:12:05.800639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:12:05.800639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:12:05.800639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:12:05.800639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:12:05.800639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:12:05.800639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:12:05.800639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:12:05.800639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:12:05.800639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:12:05.801812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:12:05.802990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:12:05.802990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:12:05.802990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:12:05.802990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 631/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:12:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:12:05.807757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4682d9-4bc1-412e-85b0-dceec327a049 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:05.904814 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4682d9-4bc1-412e-85b0-dceec327a049 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:05.936764 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0250d-463a-4eff-b708-bd45a089c35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:05.937358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0250d-463a-4eff-b708-bd45a089c35d None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:05.937664 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0250d-463a-4eff-b708-bd45a089c35d None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:05.937935 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0250d-463a-4eff-b708-bd45a089c35d None admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:05.938563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0250d-463a-4eff-b708-bd45a089c35d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:12:05.938863 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0250d-463a-4eff-b708-bd45a089c35d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:05.940238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0250d-463a-4eff-b708-bd45a089c35d None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:05.952005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 648/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:12:05.957792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfef727-2de3-47fa-ad73-61f8a3fba9be None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:05.990947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:06.059249 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfef727-2de3-47fa-ad73-61f8a3fba9be None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:06.090683 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b27b00-cb4f-40c2-8d62-d6254fe8fd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:06.091296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b27b00-cb4f-40c2-8d62-d6254fe8fd77 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:06.091597 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b27b00-cb4f-40c2-8d62-d6254fe8fd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:06.091864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b27b00-cb4f-40c2-8d62-d6254fe8fd77 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:06.098914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b27b00-cb4f-40c2-8d62-d6254fe8fd77 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:06.100365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b27b00-cb4f-40c2-8d62-d6254fe8fd77 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:06.111679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-48b27b00-cb4f-40c2-8d62-d6254fe8fd77 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 12:12:06.111679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:12:06.111679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:12:06.111679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:12:06.111679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:12:06.111679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:12:06.111679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:12:06.111679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:12:06.111679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:12:06.111679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:12:06.111679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:12:06.111679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:12:06.111679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 12:12:06.111679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 12:12:06.111679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 12:12:06.111679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:12:06.112892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:12:06.114052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:12:06.114052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:12:06.114052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 12:12:06.114052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 12:12:06.114052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 12:12:06.114052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 12:12:06.114052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 12:12:06.114052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 12:12:06.114052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 12:12:06.114052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:12:06.114052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 632/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:12:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:12:06.119168 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0250d-463a-4eff-b708-bd45a089c35d None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:06.122930 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:06.214379 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0250d-463a-4eff-b708-bd45a089c35d None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:06.247978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b54e8-194f-4302-911d-c4854c8b28c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:06.250950 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b54e8-194f-4302-911d-c4854c8b28c9 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:06.250950 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b54e8-194f-4302-911d-c4854c8b28c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:06.250950 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b54e8-194f-4302-911d-c4854c8b28c9 None admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:06.250950 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b54e8-194f-4302-911d-c4854c8b28c9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:12:06.250950 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b54e8-194f-4302-911d-c4854c8b28c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:06.251615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b54e8-194f-4302-911d-c4854c8b28c9 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:06.268455 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 649/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Oct 6 12:12:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:12:06.274459 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b27b00-cb4f-40c2-8d62-d6254fe8fd77 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:06.361749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:06.371150 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b27b00-cb4f-40c2-8d62-d6254fe8fd77 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:06.403247 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b02957-acf1-45ee-b104-f9605b35707b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:06.403864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b02957-acf1-45ee-b104-f9605b35707b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:06.404144 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b02957-acf1-45ee-b104-f9605b35707b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:06.404412 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b02957-acf1-45ee-b104-f9605b35707b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:06.467788 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b02957-acf1-45ee-b104-f9605b35707b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:06.469165 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b02957-acf1-45ee-b104-f9605b35707b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:06.554320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:06.564366 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 633/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Oct 6 12:12:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 12:12:06.570184 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b54e8-194f-4302-911d-c4854c8b28c9 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:06.629054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:06.666634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b54e8-194f-4302-911d-c4854c8b28c9 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:06.699138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90799f-25a8-4729-b27e-a544f8138176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:06.699769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90799f-25a8-4729-b27e-a544f8138176 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:06.700048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90799f-25a8-4729-b27e-a544f8138176 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:06.700317 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90799f-25a8-4729-b27e-a544f8138176 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:06.709866 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90799f-25a8-4729-b27e-a544f8138176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:06.711286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90799f-25a8-4729-b27e-a544f8138176 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:06.726092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-9a90799f-25a8-4729-b27e-a544f8138176 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:12:06.726092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:12:06.726092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:12:06.726092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:12:06.726092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:12:06.726092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:12:06.726092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:12:06.726092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:12:06.726092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:12:06.726092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:12:06.726092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:12:06.726092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:12:06.726092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 12:12:06.726092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 12:12:06.726092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 12:12:06.726092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:12:06.727269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 12:12:06.728499 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 12:12:06.728499 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 12:12:06.728499 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:12:06.728499 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 650/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:12:06.732554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b02957-acf1-45ee-b104-f9605b35707b None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:06.826712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b02957-acf1-45ee-b104-f9605b35707b None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:06.858978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2d8e9-1227-485b-8104-f298f3323a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:06.859591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2d8e9-1227-485b-8104-f298f3323a67 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:06.859874 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2d8e9-1227-485b-8104-f298f3323a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:06.860146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2d8e9-1227-485b-8104-f298f3323a67 None admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:06.860770 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2d8e9-1227-485b-8104-f298f3323a67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:12:06.861070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2d8e9-1227-485b-8104-f298f3323a67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:06.862444 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2d8e9-1227-485b-8104-f298f3323a67 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:06.873140 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 634/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Oct 6 12:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:12:06.878531 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90799f-25a8-4729-b27e-a544f8138176 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:06.980102 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90799f-25a8-4729-b27e-a544f8138176 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:06.993130 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:07.013015 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2068bb-3c5a-4523-800b-ebd2a57080e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:07.013625 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2068bb-3c5a-4523-800b-ebd2a57080e0 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:07.013901 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2068bb-3c5a-4523-800b-ebd2a57080e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:07.014165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2068bb-3c5a-4523-800b-ebd2a57080e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:07.020995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2068bb-3c5a-4523-800b-ebd2a57080e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:07.022422 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2068bb-3c5a-4523-800b-ebd2a57080e0 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:07.034501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application [None req-9c2068bb-3c5a-4523-800b-ebd2a57080e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:12:07.034501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:12:07.034501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:12:07.034501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:12:07.034501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:12:07.034501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:12:07.034501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:12:07.034501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:12:07.034501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:12:07.034501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:12:07.034501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:12:07.034501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:12:07.034501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 12:12:07.034501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 12:12:07.034501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application with Lock( Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:12:07.035599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:12:07.036906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 12:12:07.036906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 12:12:07.036906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 12:12:07.036906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 12:12:07.036906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 12:12:07.036906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 12:12:07.036906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 12:12:07.036906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: ERROR keystone.server.flask.application Oct 06 12:12:07.036906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 651/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Oct 6 12:12:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:12:07.041511 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2d8e9-1227-485b-8104-f298f3323a67 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:07.090332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:07.138343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2d8e9-1227-485b-8104-f298f3323a67 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:07.169924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c931729-1a4b-42f0-881a-c37e54d8645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:07.170516 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c931729-1a4b-42f0-881a-c37e54d8645b None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:07.170798 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c931729-1a4b-42f0-881a-c37e54d8645b None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:07.171072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c931729-1a4b-42f0-881a-c37e54d8645b None admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:07.171776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c931729-1a4b-42f0-881a-c37e54d8645b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:12:07.172081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c931729-1a4b-42f0-881a-c37e54d8645b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:07.173500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c931729-1a4b-42f0-881a-c37e54d8645b None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:07.185308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 635/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Oct 6 12:12:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:12:07.191180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2068bb-3c5a-4523-800b-ebd2a57080e0 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:07.287320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2068bb-3c5a-4523-800b-ebd2a57080e0 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:07.319308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6e19b-1b56-4b74-9079-6f350979c255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:07.320029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6e19b-1b56-4b74-9079-6f350979c255 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:07.320310 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6e19b-1b56-4b74-9079-6f350979c255 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:07.320579 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6e19b-1b56-4b74-9079-6f350979c255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:07.321879 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6e19b-1b56-4b74-9079-6f350979c255 None admin] RBAC: Adding query filter params (role.id=d5ddd52de5714fabbef9404d38ef4b35, scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f, user.id=39d67f5edf4b441ebaa2bb42e74b2cd2) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:12:07.322183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6e19b-1b56-4b74-9079-6f350979c255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:07.323631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6e19b-1b56-4b74-9079-6f350979c255 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:07.329089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 652/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Oct 6 12:12:07 2022] GET /identity/v3/role_assignments?user.id=39d67f5edf4b441ebaa2bb42e74b2cd2&scope.project.id=314d3d13ba2e4a0f93f5d3f34c50a96f&role.id=d5ddd52de5714fabbef9404d38ef4b35 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:12:09.455740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be515f8a-d501-4468-8da4-849de030218e None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:09.456293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be515f8a-d501-4468-8da4-849de030218e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:09.456293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be515f8a-d501-4468-8da4-849de030218e None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:09.457620 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 636/1288] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:12:09.463262 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed28f3-49e2-4097-ab6a-ca876a72fab7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:09.463707 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed28f3-49e2-4097-ab6a-ca876a72fab7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:09.463820 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed28f3-49e2-4097-ab6a-ca876a72fab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:09.482155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-97ed28f3-49e2-4097-ab6a-ca876a72fab7 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:12:09.483314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-97ed28f3-49e2-4097-ab6a-ca876a72fab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:12:09.513406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:09.550536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 653/1289] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:12:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:12:09.821943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be515f8a-d501-4468-8da4-849de030218e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:09.919202 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be515f8a-d501-4468-8da4-849de030218e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:09.929335 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:09.952500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af78040-763e-410a-a007-dbd5ec25af66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:09.953110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af78040-763e-410a-a007-dbd5ec25af66 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:09.953390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af78040-763e-410a-a007-dbd5ec25af66 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:09.953663 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af78040-763e-410a-a007-dbd5ec25af66 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:09.954230 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af78040-763e-410a-a007-dbd5ec25af66 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:09.955614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af78040-763e-410a-a007-dbd5ec25af66 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-2af78040-763e-410a-a007-dbd5ec25af66 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:12:09.966040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:12:09.967229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:12:09.967229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:12:09.967229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:12:09.967229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:12:09.967229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:12:09.967229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:12:09.967229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:12:09.967229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:12:09.967229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:12:09.967229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:12:09.967229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:12:09.967229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:12:09.967229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:12:09.967229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:12:09.967229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:12:09.967229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:12:09.968471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:12:09.968471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:12:09.968471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:12:09.968471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:12:09.968471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:12:09.968471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:12:09.968471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:12:09.968471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:12:09.968471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:12:09.968471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:12:09.968471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:12:09.968471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:12:09.968471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 12:12:09.968471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:12:09.968471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 637/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Oct 6 12:12:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:12:09.973740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed28f3-49e2-4097-ab6a-ca876a72fab7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:10.084718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed28f3-49e2-4097-ab6a-ca876a72fab7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:10.117141 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d399a-18f9-4c6b-b636-cfe7cbe3e089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:10.118245 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d399a-18f9-4c6b-b636-cfe7cbe3e089 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:10.118527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d399a-18f9-4c6b-b636-cfe7cbe3e089 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:10.118800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d399a-18f9-4c6b-b636-cfe7cbe3e089 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:10.119419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d399a-18f9-4c6b-b636-cfe7cbe3e089 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:12:10.119749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d399a-18f9-4c6b-b636-cfe7cbe3e089 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:10.121107 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d399a-18f9-4c6b-b636-cfe7cbe3e089 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:10.125731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 654/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Oct 6 12:12:09 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:12:10.131617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af78040-763e-410a-a007-dbd5ec25af66 None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:10.227403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af78040-763e-410a-a007-dbd5ec25af66 None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:10.260242 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ee0f1-58d4-48fa-add9-3f9cd5f12d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:10.260840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ee0f1-58d4-48fa-add9-3f9cd5f12d57 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:10.261122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ee0f1-58d4-48fa-add9-3f9cd5f12d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:10.261395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ee0f1-58d4-48fa-add9-3f9cd5f12d57 None admin] PATH_INFO: `/v3/services` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:10.262012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ee0f1-58d4-48fa-add9-3f9cd5f12d57 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:12:10.262314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ee0f1-58d4-48fa-add9-3f9cd5f12d57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:10.263702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ee0f1-58d4-48fa-add9-3f9cd5f12d57 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:10.267975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 638/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Oct 6 12:12:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:12:12.353380 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf46ef6-c1b0-4eaf-9f80-9d17e83e4699 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:12.353943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf46ef6-c1b0-4eaf-9f80-9d17e83e4699 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:12.353943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf46ef6-c1b0-4eaf-9f80-9d17e83e4699 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:12.355248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 655/1293] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:12:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:12:12.360432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2613b1-5615-43ea-8f85-86b8a04aeb25 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:12.360852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2613b1-5615-43ea-8f85-86b8a04aeb25 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:12.360977 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2613b1-5615-43ea-8f85-86b8a04aeb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:12.378513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6c2613b1-5615-43ea-8f85-86b8a04aeb25 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:12:12.379704 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6c2613b1-5615-43ea-8f85-86b8a04aeb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:12:12.408876 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:12.443965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 639/1294] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:12:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:12:12.723330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf46ef6-c1b0-4eaf-9f80-9d17e83e4699 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:12.753719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:12.821525 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf46ef6-c1b0-4eaf-9f80-9d17e83e4699 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:12.852932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71de7a9-4335-4889-b45f-29b7166894c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:12.853515 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71de7a9-4335-4889-b45f-29b7166894c4 None admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:12.853793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71de7a9-4335-4889-b45f-29b7166894c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:12.854061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71de7a9-4335-4889-b45f-29b7166894c4 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:12.854733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71de7a9-4335-4889-b45f-29b7166894c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:12.856110 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71de7a9-4335-4889-b45f-29b7166894c4 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:12.869885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 656/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Oct 6 12:12:12 2022] POST /identity/v3/services => generated 246 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:12:14.981879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff3425-b4cc-4701-9c8a-2b79f67ec757 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:14.982446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff3425-b4cc-4701-9c8a-2b79f67ec757 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:14.982538 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff3425-b4cc-4701-9c8a-2b79f67ec757 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:14.984234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 640/1296] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:12:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:12:14.990577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09e4be-c36b-4e07-8649-a75e18556077 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:14.990971 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09e4be-c36b-4e07-8649-a75e18556077 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:14.991224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09e4be-c36b-4e07-8649-a75e18556077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:15.011240 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3d09e4be-c36b-4e07-8649-a75e18556077 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:12:15.012575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3d09e4be-c36b-4e07-8649-a75e18556077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:12:15.086829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 657/1297] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:12:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:12:15.348106 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff3425-b4cc-4701-9c8a-2b79f67ec757 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:15.447377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff3425-b4cc-4701-9c8a-2b79f67ec757 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:15.480057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f906737-6d6d-4262-af67-0dd210412e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:15.480669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f906737-6d6d-4262-af67-0dd210412e2d None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:15.480953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f906737-6d6d-4262-af67-0dd210412e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:15.481228 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f906737-6d6d-4262-af67-0dd210412e2d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:15.481804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f906737-6d6d-4262-af67-0dd210412e2d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:15.483166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f906737-6d6d-4262-af67-0dd210412e2d None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-4f906737-6d6d-4262-af67-0dd210412e2d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:12:15.491748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:12:15.493032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:12:15.493032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:12:15.493032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:12:15.493032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:12:15.493032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:12:15.493032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:12:15.493032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:12:15.493032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:12:15.493032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:12:15.493032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:12:15.493032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:12:15.493032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:12:15.493032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:12:15.493032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:12:15.493032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:12:15.493032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:12:15.494069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:12:15.494069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:12:15.494069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:12:15.494069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:12:15.494069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:12:15.494069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:12:15.494069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:12:15.494069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:12:15.494069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:12:15.494069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:12:15.494069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:12:15.494069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:12:15.494069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 12:12:15.494069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:12:15.494069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 641/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Oct 6 12:12:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:12:15.498627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09e4be-c36b-4e07-8649-a75e18556077 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:15.516844 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:15.595416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09e4be-c36b-4e07-8649-a75e18556077 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:15.627520 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a224c-7850-4a65-9110-af2c862d7360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:15.628126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a224c-7850-4a65-9110-af2c862d7360 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:15.628403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a224c-7850-4a65-9110-af2c862d7360 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:15.628674 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a224c-7850-4a65-9110-af2c862d7360 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:15.629287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a224c-7850-4a65-9110-af2c862d7360 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:12:15.629585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a224c-7850-4a65-9110-af2c862d7360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:15.630935 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a224c-7850-4a65-9110-af2c862d7360 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:15.636402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 658/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Oct 6 12:12:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:12:15.642273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f906737-6d6d-4262-af67-0dd210412e2d None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:15.673333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:15.748182 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f906737-6d6d-4262-af67-0dd210412e2d None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:15.780573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7dfbdf-10ca-47f9-9100-503f7e940e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:15.781161 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dfbdf-10ca-47f9-9100-503f7e940e56 None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:15.781444 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dfbdf-10ca-47f9-9100-503f7e940e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:15.781719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dfbdf-10ca-47f9-9100-503f7e940e56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:15.782412 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7dfbdf-10ca-47f9-9100-503f7e940e56 None admin] RBAC: Adding query filter params (interface=public, service_id=a6bb83fc1ee74f4e9f8d07fbe22ffe62, region_id=RegionOne) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:12:15.782720 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7dfbdf-10ca-47f9-9100-503f7e940e56 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:15.784110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7dfbdf-10ca-47f9-9100-503f7e940e56 None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:15.788942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 642/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Oct 6 12:12:15 2022] GET /identity/v3/endpoints?service_id=a6bb83fc1ee74f4e9f8d07fbe22ffe62&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:12:15.794724 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a224c-7850-4a65-9110-af2c862d7360 None admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:15.872539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:15.890526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a224c-7850-4a65-9110-af2c862d7360 None admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:15.922934 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a97fa-5949-442a-9421-3bd502e81f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:15.923551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a97fa-5949-442a-9421-3bd502e81f86 None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:15.923836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a97fa-5949-442a-9421-3bd502e81f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:15.924109 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a97fa-5949-442a-9421-3bd502e81f86 None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:15.924703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a97fa-5949-442a-9421-3bd502e81f86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:15.926051 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a97fa-5949-442a-9421-3bd502e81f86 None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:15.933142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 659/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Oct 6 12:12:15 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:12:18.055615 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee44b62-a137-4c22-953b-c8c2b85be0d1 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:18.056156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee44b62-a137-4c22-953b-c8c2b85be0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:18.056156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee44b62-a137-4c22-953b-c8c2b85be0d1 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:18.057468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 643/1302] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:12:18.062479 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ade5c1-b35c-4b04-9409-19e87abd0591 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:18.062944 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ade5c1-b35c-4b04-9409-19e87abd0591 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:18.063049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ade5c1-b35c-4b04-9409-19e87abd0591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:18.081232 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-56ade5c1-b35c-4b04-9409-19e87abd0591 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:12:18.082405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-56ade5c1-b35c-4b04-9409-19e87abd0591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:12:18.145552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 660/1303] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:12:18 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 12:12:18.409858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee44b62-a137-4c22-953b-c8c2b85be0d1 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:18.501286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:18.508338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee44b62-a137-4c22-953b-c8c2b85be0d1 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:18.541068 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac815e-8a9b-4bea-865c-d4af89a51b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:18.541675 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac815e-8a9b-4bea-865c-d4af89a51b4a None admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:18.541954 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac815e-8a9b-4bea-865c-d4af89a51b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:18.542224 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac815e-8a9b-4bea-865c-d4af89a51b4a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:18.542792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac815e-8a9b-4bea-865c-d4af89a51b4a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:18.544170 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac815e-8a9b-4bea-865c-d4af89a51b4a None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application [None req-a0ac815e-8a9b-4bea-865c-d4af89a51b4a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 12:12:18.552944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 12:12:18.554177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 12:12:18.554177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 12:12:18.554177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 12:12:18.554177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 12:12:18.554177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 12:12:18.554177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 12:12:18.554177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 12:12:18.554177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 12:12:18.554177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 12:12:18.554177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 12:12:18.554177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 12:12:18.554177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 12:12:18.554177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 12:12:18.554177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 12:12:18.554177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 12:12:18.554177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 12:12:18.555751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 12:12:18.555751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application with Lock( Oct 06 12:12:18.555751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 12:12:18.555751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self._enter() Oct 06 12:12:18.555751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 12:12:18.555751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 12:12:18.555751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 12:12:18.555751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application return self.creator() Oct 06 12:12:18.555751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 12:12:18.555751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application created_value = creator( Oct 06 12:12:18.555751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 12:12:18.555751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 12:12:18.555751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 12:12:18.555751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: ERROR keystone.server.flask.application Oct 06 12:12:18.555751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 644/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Oct 6 12:12:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 12:12:18.560616 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ade5c1-b35c-4b04-9409-19e87abd0591 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:18.633778 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:18.657177 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ade5c1-b35c-4b04-9409-19e87abd0591 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:18.689570 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fccbfc-e97c-41ab-89ef-96fee764efbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:18.690107 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fccbfc-e97c-41ab-89ef-96fee764efbe None admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:18.690387 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fccbfc-e97c-41ab-89ef-96fee764efbe None admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:18.690659 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fccbfc-e97c-41ab-89ef-96fee764efbe None admin] PATH_INFO: `/v3/services` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:18.691277 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fccbfc-e97c-41ab-89ef-96fee764efbe None admin] RBAC: Adding query filter params (name=placement) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:12:18.691607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fccbfc-e97c-41ab-89ef-96fee764efbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:18.692965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fccbfc-e97c-41ab-89ef-96fee764efbe None admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:18.698410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 661/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Oct 6 12:12:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:12:18.704123 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac815e-8a9b-4bea-865c-d4af89a51b4a None admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:12:18.800649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac815e-8a9b-4bea-865c-d4af89a51b4a None admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:12:18.833880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a8ca8-172a-4faf-9ee1-148cf7cae72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:12:18.834465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a8ca8-172a-4faf-9ee1-148cf7cae72f None admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:12:18.834748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a8ca8-172a-4faf-9ee1-148cf7cae72f None admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:12:18.835022 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a8ca8-172a-4faf-9ee1-148cf7cae72f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:12:18.835728 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a8ca8-172a-4faf-9ee1-148cf7cae72f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:12:18.837087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a8ca8-172a-4faf-9ee1-148cf7cae72f None admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:12:18.855760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 645/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:12:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:12:28.695790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=30 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:12:28.851616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:21.994374 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132de51-2e0b-4d2d-a33e-e831e41a9e19 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:21.994886 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132de51-2e0b-4d2d-a33e-e831e41a9e19 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:21.994983 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132de51-2e0b-4d2d-a33e-e831e41a9e19 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:21.996541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 662/1307] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:13:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:13:22.003890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5112e-e0bc-40fe-ac21-9b95a8f73b47 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:22.004762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5112e-e0bc-40fe-ac21-9b95a8f73b47 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:22.005197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5112e-e0bc-40fe-ac21-9b95a8f73b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:22.098061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a0a5112e-e0bc-40fe-ac21-9b95a8f73b47 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:13:22.099528 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a0a5112e-e0bc-40fe-ac21-9b95a8f73b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:22.209492 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 646/1308] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:13:22.504244 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7c204-0d8c-4e86-90e4-c0f805df59ef None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:22.504611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7c204-0d8c-4e86-90e4-c0f805df59ef None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:22.504816 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7c204-0d8c-4e86-90e4-c0f805df59ef None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:22.506219 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 663/1309] 10.222.0.33 () {54 vars in 964 bytes} [Thu Oct 6 12:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:13:22.512754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ccedb-6764-47a8-b81d-d4c6a2563640 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:22.513205 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ccedb-6764-47a8-b81d-d4c6a2563640 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:22.513603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ccedb-6764-47a8-b81d-d4c6a2563640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:22.598252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-092ccedb-6764-47a8-b81d-d4c6a2563640 None None] MFA Rules not processed for user `2e71f9c1626f4f58916a5dd5c5987ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:13:22.599349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-092ccedb-6764-47a8-b81d-d4c6a2563640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:22.622602 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:22.683062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 647/1310] 10.222.0.33 () {58 vars in 1061 bytes} [Thu Oct 6 12:13:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:13:22.692057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7c204-0d8c-4e86-90e4-c0f805df59ef None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:13:22.694030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e7c7c204-0d8c-4e86-90e4-c0f805df59ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:22.816582 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7c204-0d8c-4e86-90e4-c0f805df59ef None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:13:22.901579 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62d84e-5425-4cf1-b734-5162c8033284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:13:22.902188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62d84e-5425-4cf1-b734-5162c8033284 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:22.902463 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62d84e-5425-4cf1-b734-5162c8033284 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:22.902732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62d84e-5425-4cf1-b734-5162c8033284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:22.904433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6d62d84e-5425-4cf1-b734-5162c8033284 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:22.961223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:23.021066 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62d84e-5425-4cf1-b734-5162c8033284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:13:23.022527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62d84e-5425-4cf1-b734-5162c8033284 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:13:23.060027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 664/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:13:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:13:25.514745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90718dcf-8cbe-4eee-a088-f9fc3213f5b7 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:25.515278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90718dcf-8cbe-4eee-a088-f9fc3213f5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:25.515278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90718dcf-8cbe-4eee-a088-f9fc3213f5b7 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:25.516644 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 648/1312] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:13:25.522130 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab3c9a-0c9b-4719-a04d-86d76afb8f34 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:25.522517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab3c9a-0c9b-4719-a04d-86d76afb8f34 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:25.522670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab3c9a-0c9b-4719-a04d-86d76afb8f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:25.603358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-86ab3c9a-0c9b-4719-a04d-86d76afb8f34 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:13:25.604600 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-86ab3c9a-0c9b-4719-a04d-86d76afb8f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:25.699708 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 665/1313] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:13:25.980663 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90718dcf-8cbe-4eee-a088-f9fc3213f5b7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:13:25.982185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-90718dcf-8cbe-4eee-a088-f9fc3213f5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:26.082649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90718dcf-8cbe-4eee-a088-f9fc3213f5b7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:13:26.151295 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:26.153428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73440227-3413-4c95-bbbd-ad857aa920c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:13:26.154135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73440227-3413-4c95-bbbd-ad857aa920c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:26.154415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73440227-3413-4c95-bbbd-ad857aa920c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:26.154692 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73440227-3413-4c95-bbbd-ad857aa920c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:26.157054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-73440227-3413-4c95-bbbd-ad857aa920c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:26.264948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73440227-3413-4c95-bbbd-ad857aa920c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:13:26.266370 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73440227-3413-4c95-bbbd-ad857aa920c8 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:13:26.284330 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 649/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:13:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:13:28.721536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e2028-53af-4643-8c57-f853663b4fea None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:28.722134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e2028-53af-4643-8c57-f853663b4fea None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:28.722134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e2028-53af-4643-8c57-f853663b4fea None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:28.723446 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 666/1315] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:13:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:13:28.728865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b9107-79ca-42e6-96f2-da39470a627e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:28.729293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b9107-79ca-42e6-96f2-da39470a627e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:28.729407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b9107-79ca-42e6-96f2-da39470a627e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:28.793871 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-b05b9107-79ca-42e6-96f2-da39470a627e None None] MFA Rules not processed for user `02b6e37825904c729c9cf6fd8e1f7dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:13:28.795005 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b05b9107-79ca-42e6-96f2-da39470a627e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:28.844847 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:28.888556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 650/1316] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:13:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:13:29.173810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e2028-53af-4643-8c57-f853663b4fea None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:13:29.175395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7b7e2028-53af-4643-8c57-f853663b4fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:29.185137 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:29.278657 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e2028-53af-4643-8c57-f853663b4fea None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:13:29.353303 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38125191-b436-4f5e-a930-56b4ba81b8fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:13:29.353914 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38125191-b436-4f5e-a930-56b4ba81b8fe service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:29.354194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38125191-b436-4f5e-a930-56b4ba81b8fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:29.354464 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38125191-b436-4f5e-a930-56b4ba81b8fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:29.356082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-38125191-b436-4f5e-a930-56b4ba81b8fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:29.417861 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:29.464768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38125191-b436-4f5e-a930-56b4ba81b8fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:13:29.466231 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38125191-b436-4f5e-a930-56b4ba81b8fe service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:13:29.484701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 667/1317] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:13:29 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:13:32.151216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5a7b7-7a4e-4963-a6dd-9979fcc59ec3 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:32.151780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5a7b7-7a4e-4963-a6dd-9979fcc59ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:32.151780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5a7b7-7a4e-4963-a6dd-9979fcc59ec3 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:32.153107 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 651/1318] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:13:32.159138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19be91a-7af6-4ef9-bfb8-8ff94a95426e None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:32.159594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19be91a-7af6-4ef9-bfb8-8ff94a95426e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:32.159777 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19be91a-7af6-4ef9-bfb8-8ff94a95426e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:32.223699 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d19be91a-7af6-4ef9-bfb8-8ff94a95426e None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:13:32.224890 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d19be91a-7af6-4ef9-bfb8-8ff94a95426e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:32.318654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 668/1319] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:13:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:13:32.602384 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5a7b7-7a4e-4963-a6dd-9979fcc59ec3 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:13:32.603951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f9f5a7b7-7a4e-4963-a6dd-9979fcc59ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:32.704682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5a7b7-7a4e-4963-a6dd-9979fcc59ec3 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:13:32.775412 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86fb8f-ccd2-4820-b423-30f1ea00cbd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:13:32.776034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86fb8f-ccd2-4820-b423-30f1ea00cbd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:32.776309 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86fb8f-ccd2-4820-b423-30f1ea00cbd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:32.776575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86fb8f-ccd2-4820-b423-30f1ea00cbd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:32.778169 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fd86fb8f-ccd2-4820-b423-30f1ea00cbd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:32.785387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:32.884946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86fb8f-ccd2-4820-b423-30f1ea00cbd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:13:32.886367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86fb8f-ccd2-4820-b423-30f1ea00cbd3 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:13:32.904262 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 652/1320] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:13:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:13:35.705998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237bb3f-6d51-476f-83ef-da0a45eb06c0 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:35.706557 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237bb3f-6d51-476f-83ef-da0a45eb06c0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:35.706557 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237bb3f-6d51-476f-83ef-da0a45eb06c0 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:35.708099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 669/1321] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:13:35.713624 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9be84-aeb9-4fd4-870f-633c15c8f4e1 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:35.714163 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9be84-aeb9-4fd4-870f-633c15c8f4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:35.714163 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9be84-aeb9-4fd4-870f-633c15c8f4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:35.787437 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-41a9be84-aeb9-4fd4-870f-633c15c8f4e1 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:13:35.788702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-41a9be84-aeb9-4fd4-870f-633c15c8f4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:35.849370 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:35.892896 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 653/1322] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:13:36.175786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237bb3f-6d51-476f-83ef-da0a45eb06c0 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:13:36.177360 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0237bb3f-6d51-476f-83ef-da0a45eb06c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:36.197633 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:36.285216 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237bb3f-6d51-476f-83ef-da0a45eb06c0 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:13:36.356890 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389a34d-7412-4202-9879-07e5228cfca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:13:36.357489 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389a34d-7412-4202-9879-07e5228cfca3 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:36.357765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389a34d-7412-4202-9879-07e5228cfca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:36.358031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389a34d-7412-4202-9879-07e5228cfca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:36.359655 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3389a34d-7412-4202-9879-07e5228cfca3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:36.429758 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:36.468649 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389a34d-7412-4202-9879-07e5228cfca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:13:36.470689 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389a34d-7412-4202-9879-07e5228cfca3 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:13:36.488903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 670/1323] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:13:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:13:38.695886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f2197-9823-4b47-88fc-0f97a6e0e803 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:38.696477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f2197-9823-4b47-88fc-0f97a6e0e803 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:38.696477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f2197-9823-4b47-88fc-0f97a6e0e803 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:38.697736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 654/1324] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:13:38.702930 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d7bbb-090d-47ad-82e2-48ca92a5e305 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:38.703368 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d7bbb-090d-47ad-82e2-48ca92a5e305 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:38.703484 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d7bbb-090d-47ad-82e2-48ca92a5e305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:38.769636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-dc9d7bbb-090d-47ad-82e2-48ca92a5e305 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:13:38.770761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-dc9d7bbb-090d-47ad-82e2-48ca92a5e305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:38.872797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 671/1325] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:13:39.158038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f2197-9823-4b47-88fc-0f97a6e0e803 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:13:39.159659 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4a9f2197-9823-4b47-88fc-0f97a6e0e803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:39.261133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f2197-9823-4b47-88fc-0f97a6e0e803 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:13:39.334772 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109dbd6-28ca-4e3b-938a-b648e2edaa29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:13:39.335379 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109dbd6-28ca-4e3b-938a-b648e2edaa29 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:39.335690 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109dbd6-28ca-4e3b-938a-b648e2edaa29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:39.335956 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109dbd6-28ca-4e3b-938a-b648e2edaa29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:39.337525 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2109dbd6-28ca-4e3b-938a-b648e2edaa29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:39.355313 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:39.450393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109dbd6-28ca-4e3b-938a-b648e2edaa29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:13:39.451776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109dbd6-28ca-4e3b-938a-b648e2edaa29 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:13:39.470426 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 655/1326] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:13:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:13:41.665796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b849-652a-42f9-9a28-992bf5b5cb45 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:41.666421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b849-652a-42f9-9a28-992bf5b5cb45 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:41.666421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b849-652a-42f9-9a28-992bf5b5cb45 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:41.667634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 672/1327] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:13:41.673278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750fcae-2539-4460-9e99-19e6baa08b12 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:41.673713 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750fcae-2539-4460-9e99-19e6baa08b12 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:41.673918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750fcae-2539-4460-9e99-19e6baa08b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:41.739266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6750fcae-2539-4460-9e99-19e6baa08b12 None None] MFA Rules not processed for user `02b6e37825904c729c9cf6fd8e1f7dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:13:41.740434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6750fcae-2539-4460-9e99-19e6baa08b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:41.804698 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:41.836634 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 656/1328] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:13:41 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:13:42.119670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4b849-652a-42f9-9a28-992bf5b5cb45 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:13:42.121249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-49d4b849-652a-42f9-9a28-992bf5b5cb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:42.154670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:42.228626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4b849-652a-42f9-9a28-992bf5b5cb45 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:13:42.308007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff99e46-b9cb-455f-b5dd-738d7004ce24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:13:42.308526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff99e46-b9cb-455f-b5dd-738d7004ce24 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:42.308806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff99e46-b9cb-455f-b5dd-738d7004ce24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:42.309074 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff99e46-b9cb-455f-b5dd-738d7004ce24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:42.310637 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-eff99e46-b9cb-455f-b5dd-738d7004ce24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:42.389153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:42.422379 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff99e46-b9cb-455f-b5dd-738d7004ce24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:13:42.423776 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff99e46-b9cb-455f-b5dd-738d7004ce24 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:13:42.441701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 673/1329] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:13:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:13:44.804537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222748b-f604-4d9f-8592-5a3cb81bd445 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:44.805112 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222748b-f604-4d9f-8592-5a3cb81bd445 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:44.805112 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222748b-f604-4d9f-8592-5a3cb81bd445 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:44.806395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 657/1330] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:13:44.811539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc2e4d-4f62-41c1-ab07-b808d2194b63 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:44.811965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc2e4d-4f62-41c1-ab07-b808d2194b63 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:44.812104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc2e4d-4f62-41c1-ab07-b808d2194b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:44.878705 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-1acc2e4d-4f62-41c1-ab07-b808d2194b63 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:13:44.879943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1acc2e4d-4f62-41c1-ab07-b808d2194b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:44.974237 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 674/1331] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:13:45.256346 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222748b-f604-4d9f-8592-5a3cb81bd445 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:13:45.257865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d222748b-f604-4d9f-8592-5a3cb81bd445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:45.359303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222748b-f604-4d9f-8592-5a3cb81bd445 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:13:45.430460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c25b38-33b1-410c-83be-94ecb4e3d71a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:13:45.431050 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c25b38-33b1-410c-83be-94ecb4e3d71a service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:45.431323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c25b38-33b1-410c-83be-94ecb4e3d71a service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:45.431618 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c25b38-33b1-410c-83be-94ecb4e3d71a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:45.433187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-45c25b38-33b1-410c-83be-94ecb4e3d71a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:45.459229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:45.539416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c25b38-33b1-410c-83be-94ecb4e3d71a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:13:45.540814 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c25b38-33b1-410c-83be-94ecb4e3d71a service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:13:45.558572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 658/1332] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:13:47.724968 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacdda7-d286-4445-a951-9a2e866dcef7 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:47.725514 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacdda7-d286-4445-a951-9a2e866dcef7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:47.725514 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacdda7-d286-4445-a951-9a2e866dcef7 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:47.726830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 675/1333] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:13:47.731968 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1f740-7765-49f3-ada6-d2d43ed4ef72 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:47.732350 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1f740-7765-49f3-ada6-d2d43ed4ef72 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:47.732521 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1f740-7765-49f3-ada6-d2d43ed4ef72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:47.797205 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2cf1f740-7765-49f3-ada6-d2d43ed4ef72 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:13:47.798408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2cf1f740-7765-49f3-ada6-d2d43ed4ef72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:47.869951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:47.893099 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 659/1334] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:13:48.179818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacdda7-d286-4445-a951-9a2e866dcef7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:13:48.181371 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-acacdda7-d286-4445-a951-9a2e866dcef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:48.222104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:48.288379 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacdda7-d286-4445-a951-9a2e866dcef7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:13:48.380186 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce26884-8c96-4fbd-afda-be105cc0715e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:13:48.380813 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce26884-8c96-4fbd-afda-be105cc0715e service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:48.381098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce26884-8c96-4fbd-afda-be105cc0715e service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:48.381374 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce26884-8c96-4fbd-afda-be105cc0715e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:48.382980 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0ce26884-8c96-4fbd-afda-be105cc0715e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:48.466630 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:48.492690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce26884-8c96-4fbd-afda-be105cc0715e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:13:48.494154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce26884-8c96-4fbd-afda-be105cc0715e service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:13:48.512791 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 676/1335] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:13:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:13:51.006057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797d039-464c-4914-910f-c2a425d4fa5f None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:51.006523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797d039-464c-4914-910f-c2a425d4fa5f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:51.006619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797d039-464c-4914-910f-c2a425d4fa5f None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:51.007982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 660/1336] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:13:51.013449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87d2ed-8237-4e49-8d0b-9af1a13dece6 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:51.013862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87d2ed-8237-4e49-8d0b-9af1a13dece6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:51.013996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87d2ed-8237-4e49-8d0b-9af1a13dece6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:51.079165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8f87d2ed-8237-4e49-8d0b-9af1a13dece6 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:13:51.080421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8f87d2ed-8237-4e49-8d0b-9af1a13dece6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:51.175007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 677/1337] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:13:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:13:51.460022 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797d039-464c-4914-910f-c2a425d4fa5f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:13:51.461583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3797d039-464c-4914-910f-c2a425d4fa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:51.565170 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797d039-464c-4914-910f-c2a425d4fa5f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:13:51.645153 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1ab68-1bc3-49f2-8c4f-59fbe4661567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:13:51.645761 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1ab68-1bc3-49f2-8c4f-59fbe4661567 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:51.646037 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1ab68-1bc3-49f2-8c4f-59fbe4661567 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:51.646302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1ab68-1bc3-49f2-8c4f-59fbe4661567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:51.647927 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-91e1ab68-1bc3-49f2-8c4f-59fbe4661567 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:51.682344 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:51.759992 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1ab68-1bc3-49f2-8c4f-59fbe4661567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:13:51.761709 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1ab68-1bc3-49f2-8c4f-59fbe4661567 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:13:51.780595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 661/1338] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:13:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:13:55.982824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee814c0c-0921-457d-a8fc-1999bc752ce3 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:55.983931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee814c0c-0921-457d-a8fc-1999bc752ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:55.983931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee814c0c-0921-457d-a8fc-1999bc752ce3 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:55.985276 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 678/1339] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:13:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:13:55.990549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6980ef2-aac3-4996-b251-d9eaa6fa012f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:55.991034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6980ef2-aac3-4996-b251-d9eaa6fa012f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:55.991134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6980ef2-aac3-4996-b251-d9eaa6fa012f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:56.056569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-b6980ef2-aac3-4996-b251-d9eaa6fa012f None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:13:56.057751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b6980ef2-aac3-4996-b251-d9eaa6fa012f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:56.143280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:56.155153 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 662/1340] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:13:56.455627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93511e12-cfb0-4138-a30f-85498c6c64a8 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:56.455917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93511e12-cfb0-4138-a30f-85498c6c64a8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:56.456197 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93511e12-cfb0-4138-a30f-85498c6c64a8 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:56.457550 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 679/1341] 10.222.0.33 () {54 vars in 964 bytes} [Thu Oct 6 12:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:13:56.463409 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7417e-c25c-4741-bfd7-f3d5abdddd69 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:56.463737 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7417e-c25c-4741-bfd7-f3d5abdddd69 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:56.464021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7417e-c25c-4741-bfd7-f3d5abdddd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:56.539791 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-11e7417e-c25c-4741-bfd7-f3d5abdddd69 None None] MFA Rules not processed for user `2e71f9c1626f4f58916a5dd5c5987ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:13:56.540917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-11e7417e-c25c-4741-bfd7-f3d5abdddd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:56.616808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 663/1342] 10.222.0.33 () {58 vars in 1061 bytes} [Thu Oct 6 12:13:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:13:56.625774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93511e12-cfb0-4138-a30f-85498c6c64a8 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:13:56.627306 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-93511e12-cfb0-4138-a30f-85498c6c64a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:56.680809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:56.740460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93511e12-cfb0-4138-a30f-85498c6c64a8 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:13:56.819949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242dc99-fa6d-47ad-be6f-29d5bf57d9ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:13:56.820563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242dc99-fa6d-47ad-be6f-29d5bf57d9ef service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:13:56.820845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242dc99-fa6d-47ad-be6f-29d5bf57d9ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:13:56.821118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242dc99-fa6d-47ad-be6f-29d5bf57d9ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:13:56.822707 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f242dc99-fa6d-47ad-be6f-29d5bf57d9ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:13:56.922754 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:13:56.939779 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242dc99-fa6d-47ad-be6f-29d5bf57d9ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:13:56.941234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242dc99-fa6d-47ad-be6f-29d5bf57d9ef service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:13:56.959562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 680/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:13:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:14:00.307803 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fc74f-c33c-45fc-b9ea-5475529df618 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:00.308357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fc74f-c33c-45fc-b9ea-5475529df618 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:00.308357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fc74f-c33c-45fc-b9ea-5475529df618 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:00.309740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 664/1344] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:00.315759 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dda74a-ae44-4cf1-9544-04d0e64f9299 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:00.316233 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dda74a-ae44-4cf1-9544-04d0e64f9299 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:00.316447 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dda74a-ae44-4cf1-9544-04d0e64f9299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:00.387995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-53dda74a-ae44-4cf1-9544-04d0e64f9299 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:00.389353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-53dda74a-ae44-4cf1-9544-04d0e64f9299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:00.492127 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 681/1345] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:14:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:00.774652 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229fc74f-c33c-45fc-b9ea-5475529df618 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:00.776221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-229fc74f-c33c-45fc-b9ea-5475529df618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:00.865203 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:00.882488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229fc74f-c33c-45fc-b9ea-5475529df618 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:00.955079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdeb617-055d-40a9-9421-accf8cc8c348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:00.955724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdeb617-055d-40a9-9421-accf8cc8c348 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:00.956006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdeb617-055d-40a9-9421-accf8cc8c348 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:00.956280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdeb617-055d-40a9-9421-accf8cc8c348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:00.957875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bfdeb617-055d-40a9-9421-accf8cc8c348 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:01.072316 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdeb617-055d-40a9-9421-accf8cc8c348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:01.073787 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdeb617-055d-40a9-9421-accf8cc8c348 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:01.091495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 665/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:14:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:14:05.372697 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485e0d7-ce92-4797-a87e-d308ca642f4e None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:05.373211 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485e0d7-ce92-4797-a87e-d308ca642f4e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:05.373843 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485e0d7-ce92-4797-a87e-d308ca642f4e None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:05.376921 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 682/1347] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:14:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:05.380827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de9575-c8a8-4d76-a325-e681a7149c76 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:05.381289 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de9575-c8a8-4d76-a325-e681a7149c76 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:05.381396 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de9575-c8a8-4d76-a325-e681a7149c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:05.397913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:05.459201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-87de9575-c8a8-4d76-a325-e681a7149c76 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:05.460590 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-87de9575-c8a8-4d76-a325-e681a7149c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:05.572898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 666/1348] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:05.854898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485e0d7-ce92-4797-a87e-d308ca642f4e None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:05.856490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6485e0d7-ce92-4797-a87e-d308ca642f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:05.914413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:05.959856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485e0d7-ce92-4797-a87e-d308ca642f4e None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:06.035716 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedce1a8-e7cd-49b8-85e5-ba233f4db411 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:06.036322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedce1a8-e7cd-49b8-85e5-ba233f4db411 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:06.036604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedce1a8-e7cd-49b8-85e5-ba233f4db411 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:06.036879 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedce1a8-e7cd-49b8-85e5-ba233f4db411 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:06.038469 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-eedce1a8-e7cd-49b8-85e5-ba233f4db411 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:06.142744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:06.146879 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedce1a8-e7cd-49b8-85e5-ba233f4db411 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:06.148349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedce1a8-e7cd-49b8-85e5-ba233f4db411 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:06.165910 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 683/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:14:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:14:11.013366 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4444b3-edf4-45a2-a88a-ae1423b0526c None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:11.013883 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4444b3-edf4-45a2-a88a-ae1423b0526c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:11.014450 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4444b3-edf4-45a2-a88a-ae1423b0526c None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:11.017169 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 667/1350] 10.222.0.33 () {54 vars in 921 bytes} [Thu Oct 6 12:14:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:11.027344 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa38ee-1595-46d2-b344-f7c1d96bc603 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:11.027344 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa38ee-1595-46d2-b344-f7c1d96bc603 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:11.027344 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa38ee-1595-46d2-b344-f7c1d96bc603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:11.133675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c1aa38ee-1595-46d2-b344-f7c1d96bc603 None None] MFA Rules not processed for user `39d67f5edf4b441ebaa2bb42e74b2cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:11.135044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c1aa38ee-1595-46d2-b344-f7c1d96bc603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:11.272463 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 684/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Oct 6 12:14:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:15.571475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=59 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:15.584366 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bdcb5-08ee-48ee-9c48-55e9d444f1ff None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:15.584675 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bdcb5-08ee-48ee-9c48-55e9d444f1ff None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:15.584920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bdcb5-08ee-48ee-9c48-55e9d444f1ff None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:15.586457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 668/1352] 10.222.0.33 () {54 vars in 921 bytes} [Thu Oct 6 12:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:15.591227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639127b-c54d-47e2-b2c2-57e9fa74f1c4 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:15.591714 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639127b-c54d-47e2-b2c2-57e9fa74f1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:15.591794 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639127b-c54d-47e2-b2c2-57e9fa74f1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:15.647994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:15.672704 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8639127b-c54d-47e2-b2c2-57e9fa74f1c4 None None] MFA Rules not processed for user `39d67f5edf4b441ebaa2bb42e74b2cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:15.673910 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8639127b-c54d-47e2-b2c2-57e9fa74f1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:15.790811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 685/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Oct 6 12:14:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:17.049677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f117547-4898-4f04-976a-bceb2673c550 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:17.050265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f117547-4898-4f04-976a-bceb2673c550 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:17.050265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f117547-4898-4f04-976a-bceb2673c550 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:17.051563 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 669/1354] 10.222.0.33 () {54 vars in 921 bytes} [Thu Oct 6 12:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:17.056544 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803f492-6f51-4502-8aac-c0078df78bc9 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:17.056971 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803f492-6f51-4502-8aac-c0078df78bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:17.057795 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803f492-6f51-4502-8aac-c0078df78bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:17.135750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c803f492-6f51-4502-8aac-c0078df78bc9 None None] MFA Rules not processed for user `39d67f5edf4b441ebaa2bb42e74b2cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:17.136968 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c803f492-6f51-4502-8aac-c0078df78bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:17.179445 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:17.247392 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 686/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Oct 6 12:14:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:14:20.766886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c327a6-4422-40d7-aa8e-00cb324944eb None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:20.767391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c327a6-4422-40d7-aa8e-00cb324944eb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:20.767391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c327a6-4422-40d7-aa8e-00cb324944eb None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:20.769282 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 670/1356] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:20.774744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d72019-475c-48a3-8fd9-91bd1d35d96c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:20.775711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d72019-475c-48a3-8fd9-91bd1d35d96c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:20.776295 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d72019-475c-48a3-8fd9-91bd1d35d96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:20.847277 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-34d72019-475c-48a3-8fd9-91bd1d35d96c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:20.848749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-34d72019-475c-48a3-8fd9-91bd1d35d96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:20.955544 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 687/1357] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:21.253684 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f42b2-a812-4b13-92ca-d201c941a9e8 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:21.254181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f42b2-a812-4b13-92ca-d201c941a9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:21.254574 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f42b2-a812-4b13-92ca-d201c941a9e8 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:21.256281 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 671/1358] 10.222.0.33 () {54 vars in 961 bytes} [Thu Oct 6 12:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:21.273839 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d1a5f-bd82-40c8-8366-f34ea0cc76b2 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:21.274498 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d1a5f-bd82-40c8-8366-f34ea0cc76b2 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:21.275005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d1a5f-bd82-40c8-8366-f34ea0cc76b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:21.309136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:21.381664 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-657d1a5f-bd82-40c8-8366-f34ea0cc76b2 None None] MFA Rules not processed for user `5a46e63d34564d34b6b5752706375798`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:21.383164 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-657d1a5f-bd82-40c8-8366-f34ea0cc76b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:21.507183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 688/1359] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Oct 6 12:14:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:21.515657 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f42b2-a812-4b13-92ca-d201c941a9e8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:21.517476 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-041f42b2-a812-4b13-92ca-d201c941a9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:21.671328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f42b2-a812-4b13-92ca-d201c941a9e8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:21.750011 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:21.791271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78c27b-496f-4b49-8cc6-4e1587a66da8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:21.791937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78c27b-496f-4b49-8cc6-4e1587a66da8 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:21.792328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78c27b-496f-4b49-8cc6-4e1587a66da8 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:21.792695 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78c27b-496f-4b49-8cc6-4e1587a66da8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:21.794363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bb78c27b-496f-4b49-8cc6-4e1587a66da8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:21.893344 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78c27b-496f-4b49-8cc6-4e1587a66da8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:21.895168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78c27b-496f-4b49-8cc6-4e1587a66da8 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:21.914489 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 672/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:14:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:14:24.272236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343dee1-9025-4feb-aa60-0f5ad8d3a949 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:24.272236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343dee1-9025-4feb-aa60-0f5ad8d3a949 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:24.272999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343dee1-9025-4feb-aa60-0f5ad8d3a949 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:24.279185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495e2be-5bc7-401f-b7f5-5873d31db0a4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:24.279890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495e2be-5bc7-401f-b7f5-5873d31db0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:24.279890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495e2be-5bc7-401f-b7f5-5873d31db0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:24.284993 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 689/1361] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:14:24 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:24.349376 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9495e2be-5bc7-401f-b7f5-5873d31db0a4 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:24.350564 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9495e2be-5bc7-401f-b7f5-5873d31db0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:24.389171 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:24.458711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 673/1362] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:24.745797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343dee1-9025-4feb-aa60-0f5ad8d3a949 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:24.747329 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c343dee1-9025-4feb-aa60-0f5ad8d3a949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:24.825689 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:24.926336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343dee1-9025-4feb-aa60-0f5ad8d3a949 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:25.072984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1884f-78ff-4303-972d-c9316a7e58da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:25.073596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1884f-78ff-4303-972d-c9316a7e58da service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:25.073876 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1884f-78ff-4303-972d-c9316a7e58da service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:25.074147 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1884f-78ff-4303-972d-c9316a7e58da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:25.075797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b2c1884f-78ff-4303-972d-c9316a7e58da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:25.098910 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:25.171250 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1884f-78ff-4303-972d-c9316a7e58da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:25.172892 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1884f-78ff-4303-972d-c9316a7e58da service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:25.191466 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 690/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:14:27.678472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b69c09-e0c2-47a4-936f-da5d607a1649 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:27.679038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b69c09-e0c2-47a4-936f-da5d607a1649 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:27.679038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b69c09-e0c2-47a4-936f-da5d607a1649 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:27.680546 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 674/1364] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:27.685662 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e851f0c-aed1-4bf2-8455-a8d1068cf253 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:27.686090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e851f0c-aed1-4bf2-8455-a8d1068cf253 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:27.686222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e851f0c-aed1-4bf2-8455-a8d1068cf253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:27.756192 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6e851f0c-aed1-4bf2-8455-a8d1068cf253 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:27.757413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6e851f0c-aed1-4bf2-8455-a8d1068cf253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:27.862558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:27.865034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 691/1365] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:28.154680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bee82-ae36-4905-b329-cda98f74a029 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:28.155080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bee82-ae36-4905-b329-cda98f74a029 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:28.155465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bee82-ae36-4905-b329-cda98f74a029 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:28.157433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 675/1366] 10.222.0.33 () {54 vars in 961 bytes} [Thu Oct 6 12:14:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:28.162983 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f15633-cdf7-4f12-9f74-831e2c55817c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:28.163376 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f15633-cdf7-4f12-9f74-831e2c55817c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:28.163902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f15633-cdf7-4f12-9f74-831e2c55817c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:28.267378 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e3f15633-cdf7-4f12-9f74-831e2c55817c None None] MFA Rules not processed for user `5a46e63d34564d34b6b5752706375798`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:28.269222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e3f15633-cdf7-4f12-9f74-831e2c55817c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:28.402410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 692/1367] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Oct 6 12:14:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:28.411128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bee82-ae36-4905-b329-cda98f74a029 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:28.412790 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9b4bee82-ae36-4905-b329-cda98f74a029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:28.564142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bee82-ae36-4905-b329-cda98f74a029 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:28.585504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:28.692414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a53dae-6bdc-4a4a-94d9-ca77687a3860 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:28.693796 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a53dae-6bdc-4a4a-94d9-ca77687a3860 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:28.694191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a53dae-6bdc-4a4a-94d9-ca77687a3860 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:28.694694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a53dae-6bdc-4a4a-94d9-ca77687a3860 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:28.696623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-96a53dae-6bdc-4a4a-94d9-ca77687a3860 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:28.796290 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a53dae-6bdc-4a4a-94d9-ca77687a3860 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:28.798047 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a53dae-6bdc-4a4a-94d9-ca77687a3860 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:28.817267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 676/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:14:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:14:28.874426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4759f6-5fec-4b42-ad0f-7771ae6bd652 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:28.874971 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4759f6-5fec-4b42-ad0f-7771ae6bd652 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:28.875204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4759f6-5fec-4b42-ad0f-7771ae6bd652 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:28.876911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 693/1369] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:28.882264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb5863-e38c-470d-be31-1bc7df524c2d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:28.882549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb5863-e38c-470d-be31-1bc7df524c2d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:28.882815 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb5863-e38c-470d-be31-1bc7df524c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:28.917764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:28.963638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ecdb5863-e38c-470d-be31-1bc7df524c2d None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:28.965167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ecdb5863-e38c-470d-be31-1bc7df524c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:29.090331 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 677/1370] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:29.392695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41505e-f621-4c33-b30e-78de83912a06 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:29.392990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41505e-f621-4c33-b30e-78de83912a06 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:29.393251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41505e-f621-4c33-b30e-78de83912a06 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:29.394844 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 694/1371] 10.222.0.33 () {54 vars in 963 bytes} [Thu Oct 6 12:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:29.401849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3121f-efca-46f3-acea-b0aa837ad2a4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:29.402376 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3121f-efca-46f3-acea-b0aa837ad2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:29.402649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3121f-efca-46f3-acea-b0aa837ad2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:29.482108 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-b8b3121f-efca-46f3-acea-b0aa837ad2a4 None None] MFA Rules not processed for user `43e873f881e6497ea09f7f10f08e3bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:29.483328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b8b3121f-efca-46f3-acea-b0aa837ad2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:29.535061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:29.567167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 678/1372] 10.222.0.33 () {58 vars in 1060 bytes} [Thu Oct 6 12:14:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:29.574720 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41505e-f621-4c33-b30e-78de83912a06 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:29.576335 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1e41505e-f621-4c33-b30e-78de83912a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:29.631862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:29.691110 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41505e-f621-4c33-b30e-78de83912a06 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:29.776749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb924d-d38b-4ee4-97ce-1be4b5279bcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:29.777368 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb924d-d38b-4ee4-97ce-1be4b5279bcd service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:29.777652 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb924d-d38b-4ee4-97ce-1be4b5279bcd service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:29.777926 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb924d-d38b-4ee4-97ce-1be4b5279bcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:29.779572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-65cb924d-d38b-4ee4-97ce-1be4b5279bcd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:29.918372 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:29.934963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb924d-d38b-4ee4-97ce-1be4b5279bcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:29.938054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb924d-d38b-4ee4-97ce-1be4b5279bcd service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:29.969547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 695/1373] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:14:31.057262 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e94646-132d-43af-92e5-7d9390c36aab None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:31.057818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e94646-132d-43af-92e5-7d9390c36aab None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:31.057818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e94646-132d-43af-92e5-7d9390c36aab None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:31.059207 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 679/1374] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:31.065095 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b29e1a-1a94-41b4-92cd-3224daaf0173 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:31.065498 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b29e1a-1a94-41b4-92cd-3224daaf0173 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:31.065648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b29e1a-1a94-41b4-92cd-3224daaf0173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:31.131647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c0b29e1a-1a94-41b4-92cd-3224daaf0173 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:31.132838 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c0b29e1a-1a94-41b4-92cd-3224daaf0173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:31.228367 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 696/1375] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:31.516399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e94646-132d-43af-92e5-7d9390c36aab None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:31.518059 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a3e94646-132d-43af-92e5-7d9390c36aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:31.671703 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e94646-132d-43af-92e5-7d9390c36aab None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:31.724755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:31.789992 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd84651-6dc4-438a-881e-1c848d9886e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:31.790598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd84651-6dc4-438a-881e-1c848d9886e7 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:31.790985 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd84651-6dc4-438a-881e-1c848d9886e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:31.791257 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd84651-6dc4-438a-881e-1c848d9886e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:31.792879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0cd84651-6dc4-438a-881e-1c848d9886e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:31.886790 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd84651-6dc4-438a-881e-1c848d9886e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:31.888574 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd84651-6dc4-438a-881e-1c848d9886e7 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:31.908011 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 680/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:14:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:14:34.118015 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af017a-4f22-42d6-9f8e-0f4e2baf38e1 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:34.118557 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af017a-4f22-42d6-9f8e-0f4e2baf38e1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:34.118557 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af017a-4f22-42d6-9f8e-0f4e2baf38e1 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:34.120086 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 697/1377] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:34.125306 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ad76a-a6a7-4507-960c-3f4ff538a68d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:34.125759 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ad76a-a6a7-4507-960c-3f4ff538a68d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:34.125844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ad76a-a6a7-4507-960c-3f4ff538a68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:34.195417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-da6ad76a-a6a7-4507-960c-3f4ff538a68d None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:34.196627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-da6ad76a-a6a7-4507-960c-3f4ff538a68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:34.203983 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:34.311552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 681/1378] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:34.594048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af017a-4f22-42d6-9f8e-0f4e2baf38e1 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:34.595773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-45af017a-4f22-42d6-9f8e-0f4e2baf38e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:34.661269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:34.751488 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af017a-4f22-42d6-9f8e-0f4e2baf38e1 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:34.868081 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f9150-0dc6-445e-b145-38fc7333d2f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:34.868833 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f9150-0dc6-445e-b145-38fc7333d2f5 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:34.869212 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f9150-0dc6-445e-b145-38fc7333d2f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:34.869584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f9150-0dc6-445e-b145-38fc7333d2f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:34.871258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-003f9150-0dc6-445e-b145-38fc7333d2f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:34.891221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:34.967808 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f9150-0dc6-445e-b145-38fc7333d2f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:34.970336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f9150-0dc6-445e-b145-38fc7333d2f5 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:34.991806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 698/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:14:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:14:37.182329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3764727-8014-451e-9e3b-6e2e40c23df0 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:37.182901 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3764727-8014-451e-9e3b-6e2e40c23df0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:37.182901 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3764727-8014-451e-9e3b-6e2e40c23df0 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:37.184387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 682/1380] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:37.190062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bafc9b-ffd6-4e2d-91d1-c0f06db5db60 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:37.190508 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bafc9b-ffd6-4e2d-91d1-c0f06db5db60 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:37.190610 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bafc9b-ffd6-4e2d-91d1-c0f06db5db60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:37.261155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c4bafc9b-ffd6-4e2d-91d1-c0f06db5db60 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:37.262379 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c4bafc9b-ffd6-4e2d-91d1-c0f06db5db60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:37.371814 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:37.391726 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 699/1381] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:37.691194 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3764727-8014-451e-9e3b-6e2e40c23df0 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:37.692766 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d3764727-8014-451e-9e3b-6e2e40c23df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:37.839200 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:37.850747 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3764727-8014-451e-9e3b-6e2e40c23df0 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:37.970836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d4861-dc53-4897-a08c-18860e92aa1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:37.971355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d4861-dc53-4897-a08c-18860e92aa1e service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:37.971686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d4861-dc53-4897-a08c-18860e92aa1e service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:37.971983 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d4861-dc53-4897-a08c-18860e92aa1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:37.973683 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a55d4861-dc53-4897-a08c-18860e92aa1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:38.073379 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d4861-dc53-4897-a08c-18860e92aa1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:38.075377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d4861-dc53-4897-a08c-18860e92aa1e service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:38.087806 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:38.098128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 683/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:14:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:14:40.317032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fd223-c2e2-4262-88e9-778b61ef0217 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:40.317717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fd223-c2e2-4262-88e9-778b61ef0217 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:40.317717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fd223-c2e2-4262-88e9-778b61ef0217 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:40.319062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 700/1383] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:40.324575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d8ddf-b4f4-45ba-84b9-a23851bc9a85 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:40.325087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d8ddf-b4f4-45ba-84b9-a23851bc9a85 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:40.325087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d8ddf-b4f4-45ba-84b9-a23851bc9a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:40.402845 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d13d8ddf-b4f4-45ba-84b9-a23851bc9a85 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:40.404091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d13d8ddf-b4f4-45ba-84b9-a23851bc9a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:40.508748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 684/1384] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:14:40.797866 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215fd223-c2e2-4262-88e9-778b61ef0217 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:40.799413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-215fd223-c2e2-4262-88e9-778b61ef0217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:40.951206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215fd223-c2e2-4262-88e9-778b61ef0217 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:41.026136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:41.074313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55949a-cf0d-4607-8262-3281893d7623 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:41.074857 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55949a-cf0d-4607-8262-3281893d7623 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:41.075092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55949a-cf0d-4607-8262-3281893d7623 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:41.075362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55949a-cf0d-4607-8262-3281893d7623 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:41.076998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ca55949a-cf0d-4607-8262-3281893d7623 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:41.167982 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55949a-cf0d-4607-8262-3281893d7623 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:41.169596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55949a-cf0d-4607-8262-3281893d7623 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:41.188702 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 701/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:14:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:14:43.786689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c9c1f-092e-4d37-ba01-fb096372df1b None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:43.787266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c9c1f-092e-4d37-ba01-fb096372df1b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:43.787266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c9c1f-092e-4d37-ba01-fb096372df1b None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:43.788519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 685/1386] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:43.793808 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d8ad1-9448-40d9-8297-edf17a7d944a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:43.794248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d8ad1-9448-40d9-8297-edf17a7d944a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:43.794365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d8ad1-9448-40d9-8297-edf17a7d944a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:43.862120 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-899d8ad1-9448-40d9-8297-edf17a7d944a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:43.863334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-899d8ad1-9448-40d9-8297-edf17a7d944a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:43.897198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:43.960925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 702/1387] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:44.252999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c9c1f-092e-4d37-ba01-fb096372df1b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:44.254530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f29c9c1f-092e-4d37-ba01-fb096372df1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:44.330096 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:44.400095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c9c1f-092e-4d37-ba01-fb096372df1b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:44.514862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b345c-3e52-4af5-a98a-519574183422 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:44.515394 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b345c-3e52-4af5-a98a-519574183422 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:44.515746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b345c-3e52-4af5-a98a-519574183422 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:44.516022 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b345c-3e52-4af5-a98a-519574183422 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:44.517605 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-685b345c-3e52-4af5-a98a-519574183422 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:44.561602 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:44.618198 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b345c-3e52-4af5-a98a-519574183422 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:44.619810 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b345c-3e52-4af5-a98a-519574183422 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:44.637775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 686/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:14:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:14:45.303927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b5e35-499c-4249-88b8-d7fc61c2f18f None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:45.304497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b5e35-499c-4249-88b8-d7fc61c2f18f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:45.304497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b5e35-499c-4249-88b8-d7fc61c2f18f None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:45.306009 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 703/1389] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:45.311053 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d9cce-5311-4d37-a19e-7610acb8571a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:45.311449 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d9cce-5311-4d37-a19e-7610acb8571a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:45.311642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d9cce-5311-4d37-a19e-7610acb8571a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:45.379494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-382d9cce-5311-4d37-a19e-7610acb8571a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:45.380722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-382d9cce-5311-4d37-a19e-7610acb8571a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:45.475158 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 687/1390] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:45.773216 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82de2be-117c-4c61-9382-df4b8b67f1ec None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:45.773508 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82de2be-117c-4c61-9382-df4b8b67f1ec None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:45.773785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82de2be-117c-4c61-9382-df4b8b67f1ec None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:45.775118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 704/1391] 10.222.0.33 () {54 vars in 962 bytes} [Thu Oct 6 12:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:45.780311 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a869e-facb-4a07-a06e-61b22bf25f98 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:45.780669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a869e-facb-4a07-a06e-61b22bf25f98 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:45.781013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a869e-facb-4a07-a06e-61b22bf25f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:45.797992 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:45.858463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-4a9a869e-facb-4a07-a06e-61b22bf25f98 None None] MFA Rules not processed for user `6a889616f86346f48b7573969174ee57`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:45.860195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4a9a869e-facb-4a07-a06e-61b22bf25f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:45.938742 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 688/1392] 10.222.0.33 () {58 vars in 1059 bytes} [Thu Oct 6 12:14:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:45.947792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82de2be-117c-4c61-9382-df4b8b67f1ec None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:45.949426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b82de2be-117c-4c61-9382-df4b8b67f1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:46.055787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82de2be-117c-4c61-9382-df4b8b67f1ec None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:46.113067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:46.128145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a484e-2ab0-4451-bca1-d027ccc051b6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a889616f86346f48b7573969174ee57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:46.128774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a484e-2ab0-4451-bca1-d027ccc051b6 service glance] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:46.129053 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a484e-2ab0-4451-bca1-d027ccc051b6 service glance] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:46.129321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a484e-2ab0-4451-bca1-d027ccc051b6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:46.130896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cf5a484e-2ab0-4451-bca1-d027ccc051b6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:46.239936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a484e-2ab0-4451-bca1-d027ccc051b6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:46.241793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a484e-2ab0-4451-bca1-d027ccc051b6 service glance] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:46.260603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 705/1393] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:14:46.807149 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df74e4-2bd1-44fa-8f54-3aef358664ea None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:46.807767 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df74e4-2bd1-44fa-8f54-3aef358664ea None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:46.807767 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df74e4-2bd1-44fa-8f54-3aef358664ea None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:46.809123 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 689/1394] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:46.814810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7b52b-fc43-4713-a5b2-f59fbb848eeb None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:46.815393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7b52b-fc43-4713-a5b2-f59fbb848eeb None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:46.815393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7b52b-fc43-4713-a5b2-f59fbb848eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:46.894712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-2ce7b52b-fc43-4713-a5b2-f59fbb848eeb None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:46.896265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2ce7b52b-fc43-4713-a5b2-f59fbb848eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:46.938202 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:46.997509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 706/1395] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:47.003581 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ee8a3-6428-4bbe-b4ed-061f0b2fb54d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:47.003813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ee8a3-6428-4bbe-b4ed-061f0b2fb54d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:47.004087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ee8a3-6428-4bbe-b4ed-061f0b2fb54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:47.072002 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a82ee8a3-6428-4bbe-b4ed-061f0b2fb54d None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:47.073214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a82ee8a3-6428-4bbe-b4ed-061f0b2fb54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:47.128674 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:47.160876 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 690/1396] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Oct 6 12:14:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:14:47.202671 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7446b-5c45-4a60-89f7-49372aea2fd5 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:47.202961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7446b-5c45-4a60-89f7-49372aea2fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:47.203796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7446b-5c45-4a60-89f7-49372aea2fd5 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:47.205161 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 707/1397] 10.222.0.33 () {54 vars in 943 bytes} [Thu Oct 6 12:14:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:47.211347 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc7522-12ee-4fea-bd39-2fd4b5c5a4d1 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:47.211672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc7522-12ee-4fea-bd39-2fd4b5c5a4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:47.211952 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc7522-12ee-4fea-bd39-2fd4b5c5a4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:47.282573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7446b-5c45-4a60-89f7-49372aea2fd5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:47.284168 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7ee7446b-5c45-4a60-89f7-49372aea2fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:47.295318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2edc7522-12ee-4fea-bd39-2fd4b5c5a4d1 None None] MFA Rules not processed for user `3deb4b248f884c19bf1e444f2fe0923a`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:47.296579 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2edc7522-12ee-4fea-bd39-2fd4b5c5a4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:47.402375 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 691/1398] 10.222.0.33 () {58 vars in 1040 bytes} [Thu Oct 6 12:14:47 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:47.412611 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc7522-12ee-4fea-bd39-2fd4b5c5a4d1 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:47.422266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc7522-12ee-4fea-bd39-2fd4b5c5a4d1 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:47.439353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7446b-5c45-4a60-89f7-49372aea2fd5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:47.448611 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:47.465061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:47.533020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42227b-d37f-4824-819a-46b89ccc2db2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:47.533548 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42227b-d37f-4824-819a-46b89ccc2db2 service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:47.533839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42227b-d37f-4824-819a-46b89ccc2db2 service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:47.534110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42227b-d37f-4824-819a-46b89ccc2db2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:47.535822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9e42227b-d37f-4824-819a-46b89ccc2db2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:47.559688 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46388aa-1a85-4b38-a3ca-72ba712173ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:47.560296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46388aa-1a85-4b38-a3ca-72ba712173ee service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:47.560579 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46388aa-1a85-4b38-a3ca-72ba712173ee service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:47.560854 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46388aa-1a85-4b38-a3ca-72ba712173ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:47.562450 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e46388aa-1a85-4b38-a3ca-72ba712173ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:47.619351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42227b-d37f-4824-819a-46b89ccc2db2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:47.620946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42227b-d37f-4824-819a-46b89ccc2db2 service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:47.629389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 692/1399] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:14:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 06 12:14:47.656324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46388aa-1a85-4b38-a3ca-72ba712173ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:47.657919 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46388aa-1a85-4b38-a3ca-72ba712173ee service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:47.676445 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 708/1400] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:14:49.872374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b976ec-3d1d-470c-ae20-ae80ffa8f6b2 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:49.872916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b976ec-3d1d-470c-ae20-ae80ffa8f6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:49.872916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b976ec-3d1d-470c-ae20-ae80ffa8f6b2 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:49.874227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 693/1401] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:49.879712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6ce56-225c-4e01-a8ed-34535f35789f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:49.880138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6ce56-225c-4e01-a8ed-34535f35789f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:49.880317 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6ce56-225c-4e01-a8ed-34535f35789f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:49.920875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:49.954164 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-dcc6ce56-225c-4e01-a8ed-34535f35789f None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:49.955402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-dcc6ce56-225c-4e01-a8ed-34535f35789f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:50.064395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 709/1402] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:50.353650 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b976ec-3d1d-470c-ae20-ae80ffa8f6b2 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:50.353650 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-94b976ec-3d1d-470c-ae20-ae80ffa8f6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:50.421003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:50.516991 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b976ec-3d1d-470c-ae20-ae80ffa8f6b2 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:50.644562 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864d61c-ec2c-48cb-a29e-6e522d63b67d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:50.645337 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864d61c-ec2c-48cb-a29e-6e522d63b67d service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:50.645866 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864d61c-ec2c-48cb-a29e-6e522d63b67d service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:50.646137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864d61c-ec2c-48cb-a29e-6e522d63b67d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:50.647754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7864d61c-ec2c-48cb-a29e-6e522d63b67d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:50.668779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:50.746934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864d61c-ec2c-48cb-a29e-6e522d63b67d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:50.748587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864d61c-ec2c-48cb-a29e-6e522d63b67d service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:50.768568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 694/1403] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:14:52.993413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ed835-afde-4a27-b57c-3bc1c4f39c28 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:52.993969 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ed835-afde-4a27-b57c-3bc1c4f39c28 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:52.993969 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ed835-afde-4a27-b57c-3bc1c4f39c28 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:52.995308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 710/1404] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:53.002064 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb389986-270e-48f4-ad51-3d56b883d72d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:53.002862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb389986-270e-48f4-ad51-3d56b883d72d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:53.003143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb389986-270e-48f4-ad51-3d56b883d72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:53.080032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-bb389986-270e-48f4-ad51-3d56b883d72d None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:53.081843 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bb389986-270e-48f4-ad51-3d56b883d72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:53.181148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:53.189198 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 695/1405] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:53.481576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ed835-afde-4a27-b57c-3bc1c4f39c28 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:53.483090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8b6ed835-afde-4a27-b57c-3bc1c4f39c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:53.601225 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:53.650309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ed835-afde-4a27-b57c-3bc1c4f39c28 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:53.781946 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d99389-eef2-490d-b874-a8b3f7b99e0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:53.782474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d99389-eef2-490d-b874-a8b3f7b99e0e service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:53.782758 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d99389-eef2-490d-b874-a8b3f7b99e0e service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:53.783047 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d99389-eef2-490d-b874-a8b3f7b99e0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:53.784666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-74d99389-eef2-490d-b874-a8b3f7b99e0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:53.867721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:53.890287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d99389-eef2-490d-b874-a8b3f7b99e0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:53.892596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d99389-eef2-490d-b874-a8b3f7b99e0e service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:53.926108 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 711/1406] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:14:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:14:56.121407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090731e-471d-4ccb-a267-2330780388a4 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:56.121950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090731e-471d-4ccb-a267-2330780388a4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:56.121950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090731e-471d-4ccb-a267-2330780388a4 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:56.123606 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 696/1407] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:14:56.129275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed599-2a43-4474-a391-406c7a169490 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:56.130053 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed599-2a43-4474-a391-406c7a169490 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:56.130338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed599-2a43-4474-a391-406c7a169490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:56.199406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-07fed599-2a43-4474-a391-406c7a169490 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:14:56.200737 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-07fed599-2a43-4474-a391-406c7a169490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:56.305649 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 712/1408] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:14:56.593855 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090731e-471d-4ccb-a267-2330780388a4 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:14:56.595435 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7090731e-471d-4ccb-a267-2330780388a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:56.756034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090731e-471d-4ccb-a267-2330780388a4 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:14:56.836800 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:14:56.895206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f810f-7476-41c3-aa21-7ece2e73fa95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:14:56.895848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f810f-7476-41c3-aa21-7ece2e73fa95 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:14:56.896385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f810f-7476-41c3-aa21-7ece2e73fa95 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:14:56.896647 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f810f-7476-41c3-aa21-7ece2e73fa95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:14:56.898383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-520f810f-7476-41c3-aa21-7ece2e73fa95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:14:57.001328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f810f-7476-41c3-aa21-7ece2e73fa95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:14:57.002959 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f810f-7476-41c3-aa21-7ece2e73fa95 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:14:57.024087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 697/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:06.304182 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=79 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:07.020822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=64 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:12.156589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01b7a9-4f48-49b8-93b6-af6bd626e699 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:12.157198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01b7a9-4f48-49b8-93b6-af6bd626e699 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:12.157198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01b7a9-4f48-49b8-93b6-af6bd626e699 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:12.158984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 713/1410] 10.222.0.33 () {54 vars in 918 bytes} [Thu Oct 6 12:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:12.164258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3efb9-2a15-4a17-9aae-dafc3c552720 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:12.164714 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3efb9-2a15-4a17-9aae-dafc3c552720 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:12.164843 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3efb9-2a15-4a17-9aae-dafc3c552720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:12.239095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9cf3efb9-2a15-4a17-9aae-dafc3c552720 None None] MFA Rules not processed for user `39d67f5edf4b441ebaa2bb42e74b2cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:12.240341 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9cf3efb9-2a15-4a17-9aae-dafc3c552720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:12.346736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 698/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Oct 6 12:15:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:12.361033 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b7aee-3b2f-4863-b2d7-42d5436724b6 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:12.361321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b7aee-3b2f-4863-b2d7-42d5436724b6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:12.361597 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b7aee-3b2f-4863-b2d7-42d5436724b6 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:12.362922 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 714/1412] 10.222.0.33 () {54 vars in 918 bytes} [Thu Oct 6 12:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:12.367529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f91eb6-4f59-4403-bbcc-fdd89a77e8c5 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:12.367843 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f91eb6-4f59-4403-bbcc-fdd89a77e8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:12.368118 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f91eb6-4f59-4403-bbcc-fdd89a77e8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:12.441808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:12.444348 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-43f91eb6-4f59-4403-bbcc-fdd89a77e8c5 None None] MFA Rules not processed for user `5a46e63d34564d34b6b5752706375798`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:12.446054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-43f91eb6-4f59-4403-bbcc-fdd89a77e8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:12.560493 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 699/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Oct 6 12:15:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:15.127880 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f58cd-aba1-4f32-8599-48cd80e7ac02 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:15.128452 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f58cd-aba1-4f32-8599-48cd80e7ac02 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:15.128452 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f58cd-aba1-4f32-8599-48cd80e7ac02 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:15.129752 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 12:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:15.135082 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd54460-fcdc-4696-bb83-ace56e8fde7a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:15.135620 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd54460-fcdc-4696-bb83-ace56e8fde7a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:15.135732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd54460-fcdc-4696-bb83-ace56e8fde7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:15.201577 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-cfd54460-fcdc-4696-bb83-ace56e8fde7a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:15.202809 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-cfd54460-fcdc-4696-bb83-ace56e8fde7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:15.257144 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:15.298436 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 12:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:15.584201 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f58cd-aba1-4f32-8599-48cd80e7ac02 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:15.585717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f35f58cd-aba1-4f32-8599-48cd80e7ac02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:15.729369 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f58cd-aba1-4f32-8599-48cd80e7ac02 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:15.800417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:15.840279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dadf3d-56b5-441c-9450-dbe4329d5880 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:15.840878 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dadf3d-56b5-441c-9450-dbe4329d5880 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:15.841159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dadf3d-56b5-441c-9450-dbe4329d5880 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:15.841429 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dadf3d-56b5-441c-9450-dbe4329d5880 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:15.843004 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-37dadf3d-56b5-441c-9450-dbe4329d5880 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:15.931226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dadf3d-56b5-441c-9450-dbe4329d5880 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:15.932834 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dadf3d-56b5-441c-9450-dbe4329d5880 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:15.950332 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 716/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:17.086867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329f7f2-fff5-4662-b8cd-5ec6bb5cf92b None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:17.087462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329f7f2-fff5-4662-b8cd-5ec6bb5cf92b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:17.087462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329f7f2-fff5-4662-b8cd-5ec6bb5cf92b None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:17.088782 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 12:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:17.094841 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61189b3-d78b-469c-bef4-aeba3e994329 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:17.095300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61189b3-d78b-469c-bef4-aeba3e994329 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:17.095443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61189b3-d78b-469c-bef4-aeba3e994329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:17.171043 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c61189b3-d78b-469c-bef4-aeba3e994329 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:17.172283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c61189b3-d78b-469c-bef4-aeba3e994329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:17.199140 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:17.270148 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 12:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:17.434786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329f7f2-fff5-4662-b8cd-5ec6bb5cf92b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:17.436333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4329f7f2-fff5-4662-b8cd-5ec6bb5cf92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:17.580905 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329f7f2-fff5-4662-b8cd-5ec6bb5cf92b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:17.621279 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:17.695216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc24023-5848-48e0-98dc-586995273b53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:17.695845 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc24023-5848-48e0-98dc-586995273b53 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:17.696128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc24023-5848-48e0-98dc-586995273b53 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:17.696400 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc24023-5848-48e0-98dc-586995273b53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:17.697992 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3fc24023-5848-48e0-98dc-586995273b53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:17.784372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc24023-5848-48e0-98dc-586995273b53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:17.785958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc24023-5848-48e0-98dc-586995273b53 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:17.803938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 702/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:18.934632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba25a7-3dd1-45a5-ae1a-f61af5a12846 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:18.935179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba25a7-3dd1-45a5-ae1a-f61af5a12846 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:18.935179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba25a7-3dd1-45a5-ae1a-f61af5a12846 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:18.936486 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 12:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:18.943321 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606a4f-fc46-4048-bf36-dbd53ea830f0 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:18.943825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606a4f-fc46-4048-bf36-dbd53ea830f0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:18.943934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606a4f-fc46-4048-bf36-dbd53ea830f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:18.997794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:19.020303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-3c606a4f-fc46-4048-bf36-dbd53ea830f0 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:19.021508 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3c606a4f-fc46-4048-bf36-dbd53ea830f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:19.116542 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 12:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:19.267526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba25a7-3dd1-45a5-ae1a-f61af5a12846 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:19.269145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e1ba25a7-3dd1-45a5-ae1a-f61af5a12846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:19.413069 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba25a7-3dd1-45a5-ae1a-f61af5a12846 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:19.425026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:19.531468 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254eb95-d487-4ffa-8aa6-3a2672383a9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:19.532017 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254eb95-d487-4ffa-8aa6-3a2672383a9a service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:19.532294 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254eb95-d487-4ffa-8aa6-3a2672383a9a service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:19.532562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254eb95-d487-4ffa-8aa6-3a2672383a9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:19.534149 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0254eb95-d487-4ffa-8aa6-3a2672383a9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:19.620833 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254eb95-d487-4ffa-8aa6-3a2672383a9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:19.622420 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254eb95-d487-4ffa-8aa6-3a2672383a9a service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:19.640541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 719/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:20.762386 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9a156-a865-4661-81c2-28c92f67a528 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:20.762919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9a156-a865-4661-81c2-28c92f67a528 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:20.762919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9a156-a865-4661-81c2-28c92f67a528 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:20.764304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 12:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:20.771135 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3b314-995c-4776-a422-97ec9e4627b0 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:20.771576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3b314-995c-4776-a422-97ec9e4627b0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:20.771710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3b314-995c-4776-a422-97ec9e4627b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:20.791975 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:20.849897 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-59f3b314-995c-4776-a422-97ec9e4627b0 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:20.851097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-59f3b314-995c-4776-a422-97ec9e4627b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:20.945589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 12:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:21.097520 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9a156-a865-4661-81c2-28c92f67a528 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:21.099028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8dc9a156-a865-4661-81c2-28c92f67a528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:21.207307 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:21.242653 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9a156-a865-4661-81c2-28c92f67a528 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:21.469548 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2506e9-76f3-491e-a9e1-ab861f3f94a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:21.470058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2506e9-76f3-491e-a9e1-ab861f3f94a6 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:21.470880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2506e9-76f3-491e-a9e1-ab861f3f94a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:21.471150 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2506e9-76f3-491e-a9e1-ab861f3f94a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:21.472871 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bc2506e9-76f3-491e-a9e1-ab861f3f94a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:21.546266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:21.560496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2506e9-76f3-491e-a9e1-ab861f3f94a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:21.562089 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2506e9-76f3-491e-a9e1-ab861f3f94a6 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:21.579387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 705/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:22.682278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab2446-590b-4d8c-98ae-95b870d608f6 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:22.682828 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab2446-590b-4d8c-98ae-95b870d608f6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:22.682943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab2446-590b-4d8c-98ae-95b870d608f6 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:22.684273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 12:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:22.691074 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327f016-be1b-4bb7-95cd-c58f4a339c35 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:22.691545 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327f016-be1b-4bb7-95cd-c58f4a339c35 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:22.691717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327f016-be1b-4bb7-95cd-c58f4a339c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:22.765751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-1327f016-be1b-4bb7-95cd-c58f4a339c35 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:22.766970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1327f016-be1b-4bb7-95cd-c58f4a339c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:22.862818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 12:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:23.011155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab2446-590b-4d8c-98ae-95b870d608f6 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:23.012670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2fab2446-590b-4d8c-98ae-95b870d608f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:23.097312 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:23.158930 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab2446-590b-4d8c-98ae-95b870d608f6 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:23.270010 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb057b-8a80-4129-9356-7e535a27d16a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:23.270553 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb057b-8a80-4129-9356-7e535a27d16a service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:23.270832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb057b-8a80-4129-9356-7e535a27d16a service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:23.271103 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb057b-8a80-4129-9356-7e535a27d16a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:23.272721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d3cb057b-8a80-4129-9356-7e535a27d16a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:23.321185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:23.362799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb057b-8a80-4129-9356-7e535a27d16a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:23.364410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb057b-8a80-4129-9356-7e535a27d16a service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:23.381856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 722/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:24.443602 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28886bd-d96a-41b1-bc36-138640e14d1b None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:24.444164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28886bd-d96a-41b1-bc36-138640e14d1b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:24.444164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28886bd-d96a-41b1-bc36-138640e14d1b None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:24.445509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 12:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:24.451630 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a69a9-f352-4bf8-8a2d-215bc1dd47f3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:24.452024 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a69a9-f352-4bf8-8a2d-215bc1dd47f3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:24.452184 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a69a9-f352-4bf8-8a2d-215bc1dd47f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:24.526850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-588a69a9-f352-4bf8-8a2d-215bc1dd47f3 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:24.528013 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-588a69a9-f352-4bf8-8a2d-215bc1dd47f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:24.623342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 12:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:24.771629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28886bd-d96a-41b1-bc36-138640e14d1b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:24.773405 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b28886bd-d96a-41b1-bc36-138640e14d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:24.823011 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:24.918210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28886bd-d96a-41b1-bc36-138640e14d1b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:25.027858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd4f0a-cfd5-4536-9ca3-13d6faa29ec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:25.028462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd4f0a-cfd5-4536-9ca3-13d6faa29ec3 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:25.028744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd4f0a-cfd5-4536-9ca3-13d6faa29ec3 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:25.029007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd4f0a-cfd5-4536-9ca3-13d6faa29ec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:25.030595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-44cd4f0a-cfd5-4536-9ca3-13d6faa29ec3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:25.037323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:25.118179 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd4f0a-cfd5-4536-9ca3-13d6faa29ec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:25.119772 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cd4f0a-cfd5-4536-9ca3-13d6faa29ec3 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:25.137467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 708/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:26.205551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da12007-9cc8-4953-9adc-d4ffb3062c7d None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:26.206133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da12007-9cc8-4953-9adc-d4ffb3062c7d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:26.206133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da12007-9cc8-4953-9adc-d4ffb3062c7d None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:26.207462 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 12:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:26.213566 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078241e-90bb-45f9-bbd3-c52f328e4eb2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:26.213999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078241e-90bb-45f9-bbd3-c52f328e4eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:26.214095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078241e-90bb-45f9-bbd3-c52f328e4eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:26.288961 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-4078241e-90bb-45f9-bbd3-c52f328e4eb2 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:26.290151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4078241e-90bb-45f9-bbd3-c52f328e4eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:26.355401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:26.383828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 12:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:26.534152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da12007-9cc8-4953-9adc-d4ffb3062c7d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:26.535644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5da12007-9cc8-4953-9adc-d4ffb3062c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:26.552801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:26.678229 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da12007-9cc8-4953-9adc-d4ffb3062c7d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:26.762725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:26.790172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25528c95-0f08-4694-b668-0109fb737bde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:26.790469 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25528c95-0f08-4694-b668-0109fb737bde service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:26.790753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25528c95-0f08-4694-b668-0109fb737bde service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:26.791123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25528c95-0f08-4694-b668-0109fb737bde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:26.792681 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-25528c95-0f08-4694-b668-0109fb737bde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:27.008345 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25528c95-0f08-4694-b668-0109fb737bde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:27.009948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25528c95-0f08-4694-b668-0109fb737bde service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:27.028827 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 725/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:28.082278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd505a-2afe-4f34-9c75-166f01a84c4d None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:28.082856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd505a-2afe-4f34-9c75-166f01a84c4d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:28.082856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd505a-2afe-4f34-9c75-166f01a84c4d None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:28.084252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 12:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:28.090380 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad502c46-8a27-4a88-8288-fb111859262a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:28.090778 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad502c46-8a27-4a88-8288-fb111859262a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:28.090943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad502c46-8a27-4a88-8288-fb111859262a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:28.165586 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ad502c46-8a27-4a88-8288-fb111859262a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:28.166800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ad502c46-8a27-4a88-8288-fb111859262a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:28.207256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:28.270619 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 12:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:28.421947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd505a-2afe-4f34-9c75-166f01a84c4d None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:28.423448 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-54bd505a-2afe-4f34-9c75-166f01a84c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:28.566664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd505a-2afe-4f34-9c75-166f01a84c4d None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:28.620918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:28.679760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40bc32-cb15-4d8b-87a4-5afb371c92c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:28.680289 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40bc32-cb15-4d8b-87a4-5afb371c92c7 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:28.680564 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40bc32-cb15-4d8b-87a4-5afb371c92c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:28.680831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40bc32-cb15-4d8b-87a4-5afb371c92c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:28.682414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2f40bc32-cb15-4d8b-87a4-5afb371c92c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:28.771892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40bc32-cb15-4d8b-87a4-5afb371c92c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:28.773754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40bc32-cb15-4d8b-87a4-5afb371c92c7 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:28.791374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 711/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:29.832567 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975793e9-c052-487b-8195-e2627b754b0e None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:29.833142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975793e9-c052-487b-8195-e2627b754b0e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:29.833142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975793e9-c052-487b-8195-e2627b754b0e None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:29.834485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 12:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:29.840886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6e3e6-2bf7-498e-b4bf-7c41ba507170 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:29.841297 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6e3e6-2bf7-498e-b4bf-7c41ba507170 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:29.841430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6e3e6-2bf7-498e-b4bf-7c41ba507170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:29.915662 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d5d6e3e6-2bf7-498e-b4bf-7c41ba507170 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:29.916876 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d5d6e3e6-2bf7-498e-b4bf-7c41ba507170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:29.918857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:30.021779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 12:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:30.172400 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975793e9-c052-487b-8195-e2627b754b0e None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:30.174439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-975793e9-c052-487b-8195-e2627b754b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:30.317851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975793e9-c052-487b-8195-e2627b754b0e None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:30.344679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:30.429491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c021fe2-fca2-4700-9853-6c16e72d3151 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:30.430024 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c021fe2-fca2-4700-9853-6c16e72d3151 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:30.430304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c021fe2-fca2-4700-9853-6c16e72d3151 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:30.430575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c021fe2-fca2-4700-9853-6c16e72d3151 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:30.432201 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1c021fe2-fca2-4700-9853-6c16e72d3151 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:30.520020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c021fe2-fca2-4700-9853-6c16e72d3151 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:30.521590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c021fe2-fca2-4700-9853-6c16e72d3151 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:30.539856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 728/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:32.595265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012dd987-c615-49c5-b949-f93508300042 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:32.596386 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012dd987-c615-49c5-b949-f93508300042 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:32.596386 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012dd987-c615-49c5-b949-f93508300042 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:32.597571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 12:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:32.603282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa6215-3b4d-4aef-9386-0a8977f0c837 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:32.603792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa6215-3b4d-4aef-9386-0a8977f0c837 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:32.603897 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa6215-3b4d-4aef-9386-0a8977f0c837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:32.644559 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:32.669881 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8efa6215-3b4d-4aef-9386-0a8977f0c837 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:32.671072 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8efa6215-3b4d-4aef-9386-0a8977f0c837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:32.764596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 12:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:33.050058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012dd987-c615-49c5-b949-f93508300042 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:33.051664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-012dd987-c615-49c5-b949-f93508300042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:33.152482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012dd987-c615-49c5-b949-f93508300042 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:33.184038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:33.225399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1161b05-ef95-4e58-aa9f-ef7f5948e5e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:33.225995 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1161b05-ef95-4e58-aa9f-ef7f5948e5e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:33.226276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1161b05-ef95-4e58-aa9f-ef7f5948e5e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:33.226538 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1161b05-ef95-4e58-aa9f-ef7f5948e5e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:33.228134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a1161b05-ef95-4e58-aa9f-ef7f5948e5e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:33.334570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1161b05-ef95-4e58-aa9f-ef7f5948e5e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:33.336082 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1161b05-ef95-4e58-aa9f-ef7f5948e5e6 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:33.353634 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 714/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:35.712949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477429c-1dd0-48be-88fb-542173a0143d None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:35.713526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477429c-1dd0-48be-88fb-542173a0143d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:35.713526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477429c-1dd0-48be-88fb-542173a0143d None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:35.714870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 12:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:35.720131 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523bf37-26d2-42b1-a947-bb0d9495c6aa None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:35.720559 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523bf37-26d2-42b1-a947-bb0d9495c6aa None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:35.720719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523bf37-26d2-42b1-a947-bb0d9495c6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:35.784874 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8523bf37-26d2-42b1-a947-bb0d9495c6aa None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:35.786065 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8523bf37-26d2-42b1-a947-bb0d9495c6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:35.799012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:35.882639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 12:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:36.168982 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477429c-1dd0-48be-88fb-542173a0143d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:36.170504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b477429c-1dd0-48be-88fb-542173a0143d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:36.273097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477429c-1dd0-48be-88fb-542173a0143d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:36.286036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:36.346884 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c27b02-9abb-4e3d-abaf-b9021a7d4b12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:36.347481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c27b02-9abb-4e3d-abaf-b9021a7d4b12 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:36.347784 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c27b02-9abb-4e3d-abaf-b9021a7d4b12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:36.348049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c27b02-9abb-4e3d-abaf-b9021a7d4b12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:36.349612 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e1c27b02-9abb-4e3d-abaf-b9021a7d4b12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:36.456827 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c27b02-9abb-4e3d-abaf-b9021a7d4b12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:36.458174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c27b02-9abb-4e3d-abaf-b9021a7d4b12 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:36.475481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 731/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:38.919022 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e4b5d-16e2-4caf-a427-d3e7d853cc31 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:38.919656 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e4b5d-16e2-4caf-a427-d3e7d853cc31 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:38.919656 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e4b5d-16e2-4caf-a427-d3e7d853cc31 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:38.920902 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 12:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:38.926681 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac040b-a4a1-46e8-a615-7f5b9767c244 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:38.927094 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac040b-a4a1-46e8-a615-7f5b9767c244 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:38.927242 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac040b-a4a1-46e8-a615-7f5b9767c244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:38.974186 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:38.993297 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b8ac040b-a4a1-46e8-a615-7f5b9767c244 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:38.994509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b8ac040b-a4a1-46e8-a615-7f5b9767c244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:39.093126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 12:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:39.374159 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e4b5d-16e2-4caf-a427-d3e7d853cc31 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:39.375763 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4e3e4b5d-16e2-4caf-a427-d3e7d853cc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:39.478502 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e4b5d-16e2-4caf-a427-d3e7d853cc31 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:39.518356 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:39.551392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bd156-d39f-418c-88cd-e21ce774a7bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:39.552120 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bd156-d39f-418c-88cd-e21ce774a7bd service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:39.552395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bd156-d39f-418c-88cd-e21ce774a7bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:39.552662 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bd156-d39f-418c-88cd-e21ce774a7bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:39.554237 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-324bd156-d39f-418c-88cd-e21ce774a7bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:39.659292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bd156-d39f-418c-88cd-e21ce774a7bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:39.660746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324bd156-d39f-418c-88cd-e21ce774a7bd service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:39.682398 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 717/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:41.959618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d72a9-1f0b-46fc-860a-c86966b615a6 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:41.960187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d72a9-1f0b-46fc-860a-c86966b615a6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:41.960187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d72a9-1f0b-46fc-860a-c86966b615a6 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:41.961494 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 12:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:41.966955 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e06fd-e1d1-4f4a-aaa0-493cd64e4697 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:41.967366 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e06fd-e1d1-4f4a-aaa0-493cd64e4697 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:41.967533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e06fd-e1d1-4f4a-aaa0-493cd64e4697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:42.032191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2f8e06fd-e1d1-4f4a-aaa0-493cd64e4697 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:42.033390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2f8e06fd-e1d1-4f4a-aaa0-493cd64e4697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:42.053756 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:42.131880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 12:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:42.417048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d72a9-1f0b-46fc-860a-c86966b615a6 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:42.418570 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8c4d72a9-1f0b-46fc-860a-c86966b615a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:42.520187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d72a9-1f0b-46fc-860a-c86966b615a6 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:42.540850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:42.592949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b3f08-ff13-49bc-8053-3019468193da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:42.593546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b3f08-ff13-49bc-8053-3019468193da service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:42.593822 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b3f08-ff13-49bc-8053-3019468193da service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:42.594088 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b3f08-ff13-49bc-8053-3019468193da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:42.595690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fe8b3f08-ff13-49bc-8053-3019468193da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:42.707573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b3f08-ff13-49bc-8053-3019468193da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:42.709005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b3f08-ff13-49bc-8053-3019468193da service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:42.727811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 734/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:44.918686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a824233-c790-40b3-8ffd-8a62ce895647 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:44.919234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a824233-c790-40b3-8ffd-8a62ce895647 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:44.919234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a824233-c790-40b3-8ffd-8a62ce895647 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:44.920604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 12:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:44.925846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b1ed2-7494-4508-9300-4ceb54fd1a3a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:44.926273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b1ed2-7494-4508-9300-4ceb54fd1a3a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:44.926380 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b1ed2-7494-4508-9300-4ceb54fd1a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:44.990638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:44.993887 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3f6b1ed2-7494-4508-9300-4ceb54fd1a3a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:44.995841 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3f6b1ed2-7494-4508-9300-4ceb54fd1a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:45.091346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 12:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:45.378748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a824233-c790-40b3-8ffd-8a62ce895647 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:45.380345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3a824233-c790-40b3-8ffd-8a62ce895647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:45.482466 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a824233-c790-40b3-8ffd-8a62ce895647 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:45.530755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:45.554552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a71dab-6c92-492e-aab2-e16f7ca16dcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:45.555152 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a71dab-6c92-492e-aab2-e16f7ca16dcb service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:45.555432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a71dab-6c92-492e-aab2-e16f7ca16dcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:45.555732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a71dab-6c92-492e-aab2-e16f7ca16dcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:45.557309 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e5a71dab-6c92-492e-aab2-e16f7ca16dcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:45.663774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a71dab-6c92-492e-aab2-e16f7ca16dcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:45.665196 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a71dab-6c92-492e-aab2-e16f7ca16dcb service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:45.682529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 720/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:47.865308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bd78f-3734-4d95-9682-b21161a1cf12 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:47.865854 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bd78f-3734-4d95-9682-b21161a1cf12 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:47.865854 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bd78f-3734-4d95-9682-b21161a1cf12 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:47.867200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 12:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:47.872556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9984ec-1b2d-413c-bdde-7641b87aab73 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:47.873004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9984ec-1b2d-413c-bdde-7641b87aab73 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:47.873139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9984ec-1b2d-413c-bdde-7641b87aab73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:47.938361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2b9984ec-1b2d-413c-bdde-7641b87aab73 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:47.939586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2b9984ec-1b2d-413c-bdde-7641b87aab73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:47.967869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:48.035494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 12:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:48.323318 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bd78f-3734-4d95-9682-b21161a1cf12 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:48.324894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f78bd78f-3734-4d95-9682-b21161a1cf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:48.429731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bd78f-3734-4d95-9682-b21161a1cf12 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:48.457407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:48.502607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77435ada-441a-4146-9192-bf9c1e2e6ab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:48.503133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77435ada-441a-4146-9192-bf9c1e2e6ab7 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:48.503410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77435ada-441a-4146-9192-bf9c1e2e6ab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:48.503709 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77435ada-441a-4146-9192-bf9c1e2e6ab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:48.505289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-77435ada-441a-4146-9192-bf9c1e2e6ab7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:48.611519 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77435ada-441a-4146-9192-bf9c1e2e6ab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:48.613522 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77435ada-441a-4146-9192-bf9c1e2e6ab7 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:48.631155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 737/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:50.818146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddea21c-f38d-42c9-b108-e8c693d655f6 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:50.818729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddea21c-f38d-42c9-b108-e8c693d655f6 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:50.818729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddea21c-f38d-42c9-b108-e8c693d655f6 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:50.820112 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 12:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:50.825336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa751e-2a09-42a6-be29-2595017160a0 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:50.825769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa751e-2a09-42a6-be29-2595017160a0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:50.825889 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa751e-2a09-42a6-be29-2595017160a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:50.891687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c0fa751e-2a09-42a6-be29-2595017160a0 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:50.893405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c0fa751e-2a09-42a6-be29-2595017160a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:50.901970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:50.987519 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 12:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:51.280298 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddea21c-f38d-42c9-b108-e8c693d655f6 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:51.281826 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7ddea21c-f38d-42c9-b108-e8c693d655f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:51.384123 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddea21c-f38d-42c9-b108-e8c693d655f6 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:51.437659 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:51.456294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c20f1-c3a5-4e23-b548-72ffcffcd34d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:51.456889 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c20f1-c3a5-4e23-b548-72ffcffcd34d service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:51.457162 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c20f1-c3a5-4e23-b548-72ffcffcd34d service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:51.457425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c20f1-c3a5-4e23-b548-72ffcffcd34d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:51.458993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0d9c20f1-c3a5-4e23-b548-72ffcffcd34d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:51.563533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c20f1-c3a5-4e23-b548-72ffcffcd34d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:51.565403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c20f1-c3a5-4e23-b548-72ffcffcd34d service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:51.583205 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 723/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:53.748705 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b09024-a5b0-4f47-875a-eb24861d8945 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:53.749282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b09024-a5b0-4f47-875a-eb24861d8945 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:53.749282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b09024-a5b0-4f47-875a-eb24861d8945 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:53.750584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 12:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:53.756382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b9bbf-f267-4bda-9c49-93dc2921904e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:53.756866 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b9bbf-f267-4bda-9c49-93dc2921904e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:53.757131 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b9bbf-f267-4bda-9c49-93dc2921904e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:53.822764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-799b9bbf-f267-4bda-9c49-93dc2921904e None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:53.824004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-799b9bbf-f267-4bda-9c49-93dc2921904e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:53.862614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:53.920749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 12:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:54.200717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b09024-a5b0-4f47-875a-eb24861d8945 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:54.202280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a5b09024-a5b0-4f47-875a-eb24861d8945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:54.306331 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b09024-a5b0-4f47-875a-eb24861d8945 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:54.344508 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:54.381965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a35e5-edf3-4dfb-a725-482509d6af70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:54.382566 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a35e5-edf3-4dfb-a725-482509d6af70 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:54.382845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a35e5-edf3-4dfb-a725-482509d6af70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:54.383117 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a35e5-edf3-4dfb-a725-482509d6af70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:54.384741 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-543a35e5-edf3-4dfb-a725-482509d6af70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:54.508154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a35e5-edf3-4dfb-a725-482509d6af70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:54.509516 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a35e5-edf3-4dfb-a725-482509d6af70 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:54.532582 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 740/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:56.740792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20e822-adc6-4fd4-aad0-5ed2fcfd44f8 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:56.741359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20e822-adc6-4fd4-aad0-5ed2fcfd44f8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:56.741475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20e822-adc6-4fd4-aad0-5ed2fcfd44f8 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:56.742773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 12:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:56.748314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14bf3d-bc22-484a-b53e-00be37f63964 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:56.748717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14bf3d-bc22-484a-b53e-00be37f63964 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:56.748859 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14bf3d-bc22-484a-b53e-00be37f63964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:56.816188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-bf14bf3d-bc22-484a-b53e-00be37f63964 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:56.817447 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bf14bf3d-bc22-484a-b53e-00be37f63964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:56.835857 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:56.913935 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 12:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:15:57.198745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20e822-adc6-4fd4-aad0-5ed2fcfd44f8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:15:57.200329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-da20e822-adc6-4fd4-aad0-5ed2fcfd44f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:57.303877 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20e822-adc6-4fd4-aad0-5ed2fcfd44f8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:15:57.369030 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:57.376008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca9776-f5d0-4bb7-a5be-310ca1fb7816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:15:57.376621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca9776-f5d0-4bb7-a5be-310ca1fb7816 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:57.376898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca9776-f5d0-4bb7-a5be-310ca1fb7816 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:57.377197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca9776-f5d0-4bb7-a5be-310ca1fb7816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:57.379303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b8ca9776-f5d0-4bb7-a5be-310ca1fb7816 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:57.485721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca9776-f5d0-4bb7-a5be-310ca1fb7816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:15:57.487080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca9776-f5d0-4bb7-a5be-310ca1fb7816 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:15:57.504740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 726/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:15:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:15:59.650060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c98bbb-862f-471a-9172-b79b6ecc84d3 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:59.650865 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c98bbb-862f-471a-9172-b79b6ecc84d3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:59.650865 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c98bbb-862f-471a-9172-b79b6ecc84d3 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:59.652226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 12:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:15:59.657816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bc6da-db6f-4195-86eb-b612b66e995f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:15:59.658225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bc6da-db6f-4195-86eb-b612b66e995f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:15:59.658378 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bc6da-db6f-4195-86eb-b612b66e995f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:15:59.723104 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8a3bc6da-db6f-4195-86eb-b612b66e995f None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:15:59.724335 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8a3bc6da-db6f-4195-86eb-b612b66e995f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:15:59.771635 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:15:59.819869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 12:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:16:00.102937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c98bbb-862f-471a-9172-b79b6ecc84d3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:16:00.104493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e6c98bbb-862f-471a-9172-b79b6ecc84d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:00.207325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c98bbb-862f-471a-9172-b79b6ecc84d3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:16:00.258493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:00.285466 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd091457-45af-428b-9780-f0bb257931a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:16:00.286069 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd091457-45af-428b-9780-f0bb257931a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:00.286351 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd091457-45af-428b-9780-f0bb257931a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:00.286625 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd091457-45af-428b-9780-f0bb257931a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:00.288260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bd091457-45af-428b-9780-f0bb257931a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:00.400459 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd091457-45af-428b-9780-f0bb257931a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:16:00.401903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd091457-45af-428b-9780-f0bb257931a4 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:16:00.420820 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 743/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:16:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:16:09.818013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=22 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:10.418375 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=66 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:20.277190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aae119-709a-4fee-aaab-88b8cbbfd22e None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:20.277667 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aae119-709a-4fee-aaab-88b8cbbfd22e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:20.277758 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aae119-709a-4fee-aaab-88b8cbbfd22e None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:20.279067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 728/1471] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:16:20.284766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea7104-741a-4c3d-a261-8e8036b1f50c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:20.285181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea7104-741a-4c3d-a261-8e8036b1f50c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:20.285286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea7104-741a-4c3d-a261-8e8036b1f50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:20.351205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f4ea7104-741a-4c3d-a261-8e8036b1f50c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:16:20.352436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f4ea7104-741a-4c3d-a261-8e8036b1f50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:20.446385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 744/1472] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:16:20.750121 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d7717-11cb-40a5-b2fd-72e5360ea1c2 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:20.750407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d7717-11cb-40a5-b2fd-72e5360ea1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:20.750679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d7717-11cb-40a5-b2fd-72e5360ea1c2 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:20.752031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 729/1473] 10.222.0.33 () {54 vars in 962 bytes} [Thu Oct 6 12:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:16:20.757119 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ffbcb-a31f-467d-bf07-284197fdb53f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:20.757406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ffbcb-a31f-467d-bf07-284197fdb53f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:20.757678 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ffbcb-a31f-467d-bf07-284197fdb53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:20.834211 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d56ffbcb-a31f-467d-bf07-284197fdb53f None None] MFA Rules not processed for user `6a889616f86346f48b7573969174ee57`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:16:20.835419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d56ffbcb-a31f-467d-bf07-284197fdb53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:20.857286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:20.912467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 745/1474] 10.222.0.33 () {58 vars in 1059 bytes} [Thu Oct 6 12:16:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:16:20.920226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d7717-11cb-40a5-b2fd-72e5360ea1c2 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:16:20.921675 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9c4d7717-11cb-40a5-b2fd-72e5360ea1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:21.026219 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d7717-11cb-40a5-b2fd-72e5360ea1c2 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:16:21.099921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad822fea-c3c9-431a-a93b-3f127e016374 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a889616f86346f48b7573969174ee57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:16:21.100529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad822fea-c3c9-431a-a93b-3f127e016374 service glance] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:21.100808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad822fea-c3c9-431a-a93b-3f127e016374 service glance] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:21.101080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad822fea-c3c9-431a-a93b-3f127e016374 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:21.102672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ad822fea-c3c9-431a-a93b-3f127e016374 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:21.155572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:21.209690 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad822fea-c3c9-431a-a93b-3f127e016374 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:16:21.211048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad822fea-c3c9-431a-a93b-3f127e016374 service glance] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:16:21.228727 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 730/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:16:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:16:23.458937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55522982-9b6f-4e7d-96fa-6147bf83cd8c None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:23.459421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55522982-9b6f-4e7d-96fa-6147bf83cd8c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:23.459571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55522982-9b6f-4e7d-96fa-6147bf83cd8c None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:23.460866 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 746/1476] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:16:23.466076 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe89d06-4a4d-453b-b259-3b9c3b5fb588 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:23.466525 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe89d06-4a4d-453b-b259-3b9c3b5fb588 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:23.466647 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe89d06-4a4d-453b-b259-3b9c3b5fb588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:23.532371 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8fe89d06-4a4d-453b-b259-3b9c3b5fb588 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:16:23.533567 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8fe89d06-4a4d-453b-b259-3b9c3b5fb588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:23.627888 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 731/1477] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:16:23.901952 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55522982-9b6f-4e7d-96fa-6147bf83cd8c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:16:23.903548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-55522982-9b6f-4e7d-96fa-6147bf83cd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:24.028564 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55522982-9b6f-4e7d-96fa-6147bf83cd8c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:16:24.076860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:24.119601 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74bd45-abf7-481c-8e27-a76f04d30e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:16:24.120219 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74bd45-abf7-481c-8e27-a76f04d30e35 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:24.120496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74bd45-abf7-481c-8e27-a76f04d30e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:24.120760 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74bd45-abf7-481c-8e27-a76f04d30e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:24.121383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74bd45-abf7-481c-8e27-a76f04d30e35 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:16:24.123100 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: WARNING py.warnings [None req-4f74bd45-abf7-481c-8e27-a76f04d30e35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 12:16:24.123100 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: warnings.warn(msg) Oct 06 12:16:24.123100 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: Oct 06 12:16:24.124255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74bd45-abf7-481c-8e27-a76f04d30e35 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:16:24.143263 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 747/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Oct 6 12:16:23 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:16:26.278385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a779201-f15c-43e3-a323-c9fe195137fb None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:26.278912 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a779201-f15c-43e3-a323-c9fe195137fb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:26.278912 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a779201-f15c-43e3-a323-c9fe195137fb None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:26.280341 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 732/1479] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:16:26.285579 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f135b67-d8d7-43b1-94f5-40978ca38a0a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:26.286013 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f135b67-d8d7-43b1-94f5-40978ca38a0a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:26.286152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f135b67-d8d7-43b1-94f5-40978ca38a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:26.351864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-7f135b67-d8d7-43b1-94f5-40978ca38a0a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:16:26.353078 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7f135b67-d8d7-43b1-94f5-40978ca38a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:26.450886 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 748/1480] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:16:26.744055 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a779201-f15c-43e3-a323-c9fe195137fb None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:16:26.745583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1a779201-f15c-43e3-a323-c9fe195137fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:26.755773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:26.891720 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a779201-f15c-43e3-a323-c9fe195137fb None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:16:26.969350 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:27.002210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bc63d-12dc-4fe0-9f50-2897b49efe34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:16:27.002713 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bc63d-12dc-4fe0-9f50-2897b49efe34 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:27.003488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bc63d-12dc-4fe0-9f50-2897b49efe34 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:27.003790 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bc63d-12dc-4fe0-9f50-2897b49efe34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:27.005378 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f96bc63d-12dc-4fe0-9f50-2897b49efe34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:27.093642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96bc63d-12dc-4fe0-9f50-2897b49efe34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:16:27.095253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96bc63d-12dc-4fe0-9f50-2897b49efe34 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:16:27.112734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 733/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:16:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:16:29.434185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e2ded-ca0c-4760-87c6-4eac67571b9c None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:29.434750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e2ded-ca0c-4760-87c6-4eac67571b9c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:29.434750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e2ded-ca0c-4760-87c6-4eac67571b9c None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:29.436098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 749/1482] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:16:29.441820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd125c79-bccc-4cc7-bab5-c7e5483a1e97 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:29.444597 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd125c79-bccc-4cc7-bab5-c7e5483a1e97 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:29.444597 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd125c79-bccc-4cc7-bab5-c7e5483a1e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:29.508765 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-fd125c79-bccc-4cc7-bab5-c7e5483a1e97 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:16:29.509893 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fd125c79-bccc-4cc7-bab5-c7e5483a1e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:29.556033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:29.606045 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 734/1483] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:16:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:16:29.895368 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e2ded-ca0c-4760-87c6-4eac67571b9c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:16:29.896933 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-672e2ded-ca0c-4760-87c6-4eac67571b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:29.920974 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:30.002884 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e2ded-ca0c-4760-87c6-4eac67571b9c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:16:30.075462 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76301e52-64dc-46f9-af66-f22903ab59db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:16:30.076080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76301e52-64dc-46f9-af66-f22903ab59db service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:30.076359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76301e52-64dc-46f9-af66-f22903ab59db service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:30.076630 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76301e52-64dc-46f9-af66-f22903ab59db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:30.078239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-76301e52-64dc-46f9-af66-f22903ab59db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:30.158694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:30.194584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76301e52-64dc-46f9-af66-f22903ab59db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:16:30.196049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76301e52-64dc-46f9-af66-f22903ab59db service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:16:30.213724 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 750/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:16:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:16:32.400999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967b87f-05ea-47c2-8ad7-4d323fb3f680 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:32.401563 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967b87f-05ea-47c2-8ad7-4d323fb3f680 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:32.401563 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967b87f-05ea-47c2-8ad7-4d323fb3f680 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:32.403195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 735/1485] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:16:32.408576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c9867-7f97-437b-8f8d-50e1ab97b2d1 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:32.409013 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c9867-7f97-437b-8f8d-50e1ab97b2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:32.409141 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c9867-7f97-437b-8f8d-50e1ab97b2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:32.474211 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-489c9867-7f97-437b-8f8d-50e1ab97b2d1 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:16:32.475413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-489c9867-7f97-437b-8f8d-50e1ab97b2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:32.569330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 751/1486] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:16:32.860700 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967b87f-05ea-47c2-8ad7-4d323fb3f680 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:16:32.862271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5967b87f-05ea-47c2-8ad7-4d323fb3f680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:32.967299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967b87f-05ea-47c2-8ad7-4d323fb3f680 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:16:33.039600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:33.041219 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d1b0e-9aca-4835-bea3-c15460012273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:16:33.041862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d1b0e-9aca-4835-bea3-c15460012273 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:33.042162 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d1b0e-9aca-4835-bea3-c15460012273 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:33.042439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d1b0e-9aca-4835-bea3-c15460012273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:33.044660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c04d1b0e-9aca-4835-bea3-c15460012273 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:33.150097 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d1b0e-9aca-4835-bea3-c15460012273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:16:33.151453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d1b0e-9aca-4835-bea3-c15460012273 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:16:33.169404 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 736/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:16:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:16:35.392614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686580b-cc37-45ba-ab0d-9119d0cbab2f None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:35.393189 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686580b-cc37-45ba-ab0d-9119d0cbab2f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:35.393189 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686580b-cc37-45ba-ab0d-9119d0cbab2f None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:35.395172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 752/1488] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:16:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:16:35.400717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459748e-eee2-4e4e-bc90-a40efbc46fa8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:35.401176 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459748e-eee2-4e4e-bc90-a40efbc46fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:35.401283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459748e-eee2-4e4e-bc90-a40efbc46fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:35.466806 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a459748e-eee2-4e4e-bc90-a40efbc46fa8 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:16:35.468036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a459748e-eee2-4e4e-bc90-a40efbc46fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:35.518181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:35.562436 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 737/1489] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:16:35.850840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686580b-cc37-45ba-ab0d-9119d0cbab2f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:16:35.852421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7686580b-cc37-45ba-ab0d-9119d0cbab2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:35.888649 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:35.955446 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686580b-cc37-45ba-ab0d-9119d0cbab2f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:16:36.029275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933133a4-9eac-40f8-b7de-5eb7d5ffd298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:16:36.029873 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933133a4-9eac-40f8-b7de-5eb7d5ffd298 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:36.030154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933133a4-9eac-40f8-b7de-5eb7d5ffd298 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:36.030425 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933133a4-9eac-40f8-b7de-5eb7d5ffd298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:36.032034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-933133a4-9eac-40f8-b7de-5eb7d5ffd298 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:36.112819 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:36.141470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933133a4-9eac-40f8-b7de-5eb7d5ffd298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:16:36.142918 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933133a4-9eac-40f8-b7de-5eb7d5ffd298 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:16:36.161329 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 753/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:16:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:16:38.486853 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67f5da-9ddd-45b3-8773-eb9e5adbe2f7 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:38.487462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67f5da-9ddd-45b3-8773-eb9e5adbe2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:38.487462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67f5da-9ddd-45b3-8773-eb9e5adbe2f7 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:38.488858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 738/1491] 10.222.0.33 () {54 vars in 927 bytes} [Thu Oct 6 12:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:16:38.494252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ce1dd-0a88-4a84-993b-6073b09f0eca None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:38.494676 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ce1dd-0a88-4a84-993b-6073b09f0eca None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:38.494784 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ce1dd-0a88-4a84-993b-6073b09f0eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:38.564850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-a34ce1dd-0a88-4a84-993b-6073b09f0eca None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:16:38.566639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a34ce1dd-0a88-4a84-993b-6073b09f0eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:38.660437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 754/1492] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Oct 6 12:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:16:38.948626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67f5da-9ddd-45b3-8773-eb9e5adbe2f7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:16:38.950171 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9d67f5da-9ddd-45b3-8773-eb9e5adbe2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:39.052092 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67f5da-9ddd-45b3-8773-eb9e5adbe2f7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:16:39.124343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c08c91-51a7-488c-9e70-c8a652d262e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:16:39.124944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c08c91-51a7-488c-9e70-c8a652d262e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:16:39.125227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c08c91-51a7-488c-9e70-c8a652d262e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:16:39.125495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c08c91-51a7-488c-9e70-c8a652d262e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:16:39.127084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-28c08c91-51a7-488c-9e70-c8a652d262e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:16:39.135913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:39.233128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c08c91-51a7-488c-9e70-c8a652d262e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:16:39.234471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c08c91-51a7-488c-9e70-c8a652d262e7 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:16:39.252429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 739/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:16:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:16:48.658873 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=82 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:16:49.250159 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=50 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:18:04.140542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7b11d-8800-4080-91ca-dfcc0649b1b8 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:04.141133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7b11d-8800-4080-91ca-dfcc0649b1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:04.141133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7b11d-8800-4080-91ca-dfcc0649b1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:04.216723 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-58a7b11d-8800-4080-91ca-dfcc0649b1b8 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:18:04.217940 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-58a7b11d-8800-4080-91ca-dfcc0649b1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:18:04.313031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 755/1494] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:18:04.321393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300872bb-8083-4f60-a815-640f409ff351 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:04.321906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300872bb-8083-4f60-a815-640f409ff351 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:04.322013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300872bb-8083-4f60-a815-640f409ff351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:04.395312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-300872bb-8083-4f60-a815-640f409ff351 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:18:04.396471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-300872bb-8083-4f60-a815-640f409ff351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:18:04.490203 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 740/1495] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:18:04.494784 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7b11d-8800-4080-91ca-dfcc0649b1b8 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:18:04.496030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-58a7b11d-8800-4080-91ca-dfcc0649b1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:18:04.507766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7b11d-8800-4080-91ca-dfcc0649b1b8 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:18:04.568539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:18:04.602952 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a583899-8fa4-496b-ab07-cf2d164db374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:18:04.603586 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a583899-8fa4-496b-ab07-cf2d164db374 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:04.603871 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a583899-8fa4-496b-ab07-cf2d164db374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:04.604142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a583899-8fa4-496b-ab07-cf2d164db374 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:04.604780 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a583899-8fa4-496b-ab07-cf2d164db374 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:18:04.605081 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a583899-8fa4-496b-ab07-cf2d164db374 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:18:04.606087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: WARNING py.warnings [None req-4a583899-8fa4-496b-ab07-cf2d164db374 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 06 12:18:04.606087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: warnings.warn(msg) Oct 06 12:18:04.606087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: Oct 06 12:18:04.606996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a583899-8fa4-496b-ab07-cf2d164db374 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:18:04.618845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 756/1496] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:18:04.627030 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300872bb-8083-4f60-a815-640f409ff351 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:18:04.636602 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300872bb-8083-4f60-a815-640f409ff351 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:18:04.705362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:18:04.738975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad1f15-0218-4341-94be-d10b3ccfb5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:18:04.739603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad1f15-0218-4341-94be-d10b3ccfb5ca admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:04.739883 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad1f15-0218-4341-94be-d10b3ccfb5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:04.740151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad1f15-0218-4341-94be-d10b3ccfb5ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:04.740781 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad1f15-0218-4341-94be-d10b3ccfb5ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:18:04.742502 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: WARNING py.warnings [None req-61ad1f15-0218-4341-94be-d10b3ccfb5ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 12:18:04.742502 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: warnings.warn(msg) Oct 06 12:18:04.742502 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: Oct 06 12:18:04.743711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad1f15-0218-4341-94be-d10b3ccfb5ca admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:18:04.799831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 741/1497] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:18:04 2022] POST /identity/v3/projects => generated 379 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:18:04.804140 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a583899-8fa4-496b-ab07-cf2d164db374 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:18:04.805687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4a583899-8fa4-496b-ab07-cf2d164db374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:18:04.839168 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a583899-8fa4-496b-ab07-cf2d164db374 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:18:04.932690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6903a6-75aa-44b6-ac06-6e91ca707adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:18:04.933278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6903a6-75aa-44b6-ac06-6e91ca707adc admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:04.933560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6903a6-75aa-44b6-ac06-6e91ca707adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:04.933825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6903a6-75aa-44b6-ac06-6e91ca707adc admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:04.934433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6903a6-75aa-44b6-ac06-6e91ca707adc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:18:04.935437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: WARNING py.warnings [None req-ba6903a6-75aa-44b6-ac06-6e91ca707adc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 12:18:04.935437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: warnings.warn(msg) Oct 06 12:18:04.935437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: Oct 06 12:18:04.936668 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6903a6-75aa-44b6-ac06-6e91ca707adc admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:18:04.978766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 757/1498] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:18:04 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:18:04.982892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad1f15-0218-4341-94be-d10b3ccfb5ca admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:18:04.984419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-61ad1f15-0218-4341-94be-d10b3ccfb5ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:18:05.047204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad1f15-0218-4341-94be-d10b3ccfb5ca admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:18:05.128042 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:18:05.139314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a3bc1-158b-42c3-aef5-98ff07c509ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:18:05.139970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a3bc1-158b-42c3-aef5-98ff07c509ca admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:05.140251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a3bc1-158b-42c3-aef5-98ff07c509ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:05.140522 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a3bc1-158b-42c3-aef5-98ff07c509ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:05.141160 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a3bc1-158b-42c3-aef5-98ff07c509ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:18:05.142167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: WARNING py.warnings [None req-907a3bc1-158b-42c3-aef5-98ff07c509ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 06 12:18:05.142167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: warnings.warn(msg) Oct 06 12:18:05.142167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: Oct 06 12:18:05.143609 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a3bc1-158b-42c3-aef5-98ff07c509ca admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:18:05.155885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 742/1499] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:18:05.159978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6903a6-75aa-44b6-ac06-6e91ca707adc admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:18:05.161493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ba6903a6-75aa-44b6-ac06-6e91ca707adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:18:05.172773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6903a6-75aa-44b6-ac06-6e91ca707adc admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:18:05.197721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:18:05.267938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a553f-d5f7-4220-8c6c-9f0414a605a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:18:05.268549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a553f-d5f7-4220-8c6c-9f0414a605a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:05.268829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a553f-d5f7-4220-8c6c-9f0414a605a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:05.269098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a553f-d5f7-4220-8c6c-9f0414a605a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:05.269731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a553f-d5f7-4220-8c6c-9f0414a605a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:18:05.270745 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: WARNING py.warnings [None req-824a553f-d5f7-4220-8c6c-9f0414a605a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 06 12:18:05.270745 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: warnings.warn(msg) Oct 06 12:18:05.270745 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: Oct 06 12:18:05.271656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a553f-d5f7-4220-8c6c-9f0414a605a1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:18:05.284621 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 758/1500] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:18:05.288779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a3bc1-158b-42c3-aef5-98ff07c509ca admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:18:05.290386 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-907a3bc1-158b-42c3-aef5-98ff07c509ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:18:05.323555 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a3bc1-158b-42c3-aef5-98ff07c509ca admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:18:05.414743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001a0a2-9c94-413d-8287-29a1b21a6081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:18:05.415395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001a0a2-9c94-413d-8287-29a1b21a6081 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:05.415700 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001a0a2-9c94-413d-8287-29a1b21a6081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:05.415974 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001a0a2-9c94-413d-8287-29a1b21a6081 admin admin] PATH_INFO: `/v3/projects/bcd37543738f4414be68c94540baa0d3/users/7158be97a7dd47c4a7583e496323cef1/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:05.437745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001a0a2-9c94-413d-8287-29a1b21a6081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcd37543738f4414be68c94540baa0d3, user_id=7158be97a7dd47c4a7583e496323cef1, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:18:05.438764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: WARNING py.warnings [None req-1001a0a2-9c94-413d-8287-29a1b21a6081 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 12:18:05.438764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: warnings.warn(msg) Oct 06 12:18:05.438764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: Oct 06 12:18:05.440743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001a0a2-9c94-413d-8287-29a1b21a6081 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:18:05.464430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 743/1501] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:18:05 2022] PUT /identity/v3/projects/bcd37543738f4414be68c94540baa0d3/users/7158be97a7dd47c4a7583e496323cef1/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:18:05.477138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248515d6-376e-44e9-bc5e-1383a8618ecf None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:05.477426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248515d6-376e-44e9-bc5e-1383a8618ecf None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:05.477702 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248515d6-376e-44e9-bc5e-1383a8618ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:05.533697 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-248515d6-376e-44e9-bc5e-1383a8618ecf None None] MFA Rules not processed for user `7158be97a7dd47c4a7583e496323cef1`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:18:05.535416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-248515d6-376e-44e9-bc5e-1383a8618ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:18:05.610794 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 759/1502] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:18:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:18:05.623965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076b742-ca61-4552-bbde-d65fb24b29bd None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:05.624254 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076b742-ca61-4552-bbde-d65fb24b29bd None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:05.624531 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076b742-ca61-4552-bbde-d65fb24b29bd None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:05.625852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 744/1503] 10.222.0.33 () {54 vars in 963 bytes} [Thu Oct 6 12:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:18:05.630442 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab64a50-f343-4c02-89c3-43f63db6e077 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:05.630727 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab64a50-f343-4c02-89c3-43f63db6e077 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:05.630997 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab64a50-f343-4c02-89c3-43f63db6e077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:05.662503 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:18:05.707651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3ab64a50-f343-4c02-89c3-43f63db6e077 None None] MFA Rules not processed for user `43e873f881e6497ea09f7f10f08e3bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:18:05.708856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3ab64a50-f343-4c02-89c3-43f63db6e077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:18:05.782005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 760/1504] 10.222.0.33 () {58 vars in 1060 bytes} [Thu Oct 6 12:18:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:18:05.791706 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076b742-ca61-4552-bbde-d65fb24b29bd None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:18:05.793289 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1076b742-ca61-4552-bbde-d65fb24b29bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:18:05.842431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:18:05.894514 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076b742-ca61-4552-bbde-d65fb24b29bd None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:18:05.967675 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0291a-9dc8-4eb0-9725-348d45e60c09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:18:05.968276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0291a-9dc8-4eb0-9725-348d45e60c09 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:05.968553 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0291a-9dc8-4eb0-9725-348d45e60c09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:05.968821 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0291a-9dc8-4eb0-9725-348d45e60c09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:05.970416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7ff0291a-9dc8-4eb0-9725-348d45e60c09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:18:06.070711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0291a-9dc8-4eb0-9725-348d45e60c09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:18:06.072119 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0291a-9dc8-4eb0-9725-348d45e60c09 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:18:06.075563 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:18:06.090411 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 745/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:18:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:18:06.135885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e2058-f416-4e32-8d84-6dbda9467cea None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:06.136168 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e2058-f416-4e32-8d84-6dbda9467cea None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:06.136460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e2058-f416-4e32-8d84-6dbda9467cea None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:06.137739 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 761/1506] 10.222.0.33 () {52 vars in 830 bytes} [Thu Oct 6 12:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:18:06.141348 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0291a-9dc8-4eb0-9725-348d45e60c09 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:18:06.142552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7ff0291a-9dc8-4eb0-9725-348d45e60c09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:18:06.240395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0291a-9dc8-4eb0-9725-348d45e60c09 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:18:06.333254 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fafd97-e755-405b-b5c1-d3ecc2f69f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:18:06.333911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fafd97-e755-405b-b5c1-d3ecc2f69f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:06.334073 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fafd97-e755-405b-b5c1-d3ecc2f69f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:06.334345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fafd97-e755-405b-b5c1-d3ecc2f69f8e admin admin] PATH_INFO: `/v3/users/7158be97a7dd47c4a7583e496323cef1` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:06.347551 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fafd97-e755-405b-b5c1-d3ecc2f69f8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7158be97a7dd47c4a7583e496323cef1)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:18:06.348574 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: WARNING py.warnings [None req-47fafd97-e755-405b-b5c1-d3ecc2f69f8e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 12:18:06.348574 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: warnings.warn(msg) Oct 06 12:18:06.348574 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: Oct 06 12:18:06.349815 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fafd97-e755-405b-b5c1-d3ecc2f69f8e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:18:06.359407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:18:06.414928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-47fafd97-e755-405b-b5c1-d3ecc2f69f8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7158be97a7dd47c4a7583e496323cef1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:18:06.428130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-47fafd97-e755-405b-b5c1-d3ecc2f69f8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7158be97a7dd47c4a7583e496323cef1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:18:06.428573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-47fafd97-e755-405b-b5c1-d3ecc2f69f8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7158be97a7dd47c4a7583e496323cef1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:18:06.429095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-47fafd97-e755-405b-b5c1-d3ecc2f69f8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7158be97a7dd47c4a7583e496323cef1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:18:06.438672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-47fafd97-e755-405b-b5c1-d3ecc2f69f8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7158be97a7dd47c4a7583e496323cef1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:18:06.466149 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 746/1507] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:18:06 2022] DELETE /identity/v3/users/7158be97a7dd47c4a7583e496323cef1 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:18:06.470528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6390d-77c3-46b5-b4b2-dfc02cf29a7c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:06.471011 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6390d-77c3-46b5-b4b2-dfc02cf29a7c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:06.471681 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6390d-77c3-46b5-b4b2-dfc02cf29a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:06.527409 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-bcf6390d-77c3-46b5-b4b2-dfc02cf29a7c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:18:06.528631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bcf6390d-77c3-46b5-b4b2-dfc02cf29a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:18:06.602653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:18:06.615061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 762/1508] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:18:06.624597 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fafd97-e755-405b-b5c1-d3ecc2f69f8e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:18:06.625817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-47fafd97-e755-405b-b5c1-d3ecc2f69f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:18:06.725863 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fafd97-e755-405b-b5c1-d3ecc2f69f8e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:18:06.779419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:18:06.802524 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913a2a7-8c85-4cee-a04e-018e80dbcbf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:18:06.803143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913a2a7-8c85-4cee-a04e-018e80dbcbf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:06.803425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913a2a7-8c85-4cee-a04e-018e80dbcbf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:06.803730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913a2a7-8c85-4cee-a04e-018e80dbcbf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:06.805332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f913a2a7-8c85-4cee-a04e-018e80dbcbf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:18:06.912633 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913a2a7-8c85-4cee-a04e-018e80dbcbf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:18:06.914091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913a2a7-8c85-4cee-a04e-018e80dbcbf8 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:18:06.932411 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 747/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:18:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:18:07.287615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6390d-77c3-46b5-b4b2-dfc02cf29a7c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:18:07.288866 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bcf6390d-77c3-46b5-b4b2-dfc02cf29a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:18:07.300720 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6390d-77c3-46b5-b4b2-dfc02cf29a7c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:18:07.399221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada36438-b9c7-4a3e-b05a-937878ff1605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:18:07.399895 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada36438-b9c7-4a3e-b05a-937878ff1605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:18:07.400170 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada36438-b9c7-4a3e-b05a-937878ff1605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:18:07.400439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada36438-b9c7-4a3e-b05a-937878ff1605 admin admin] PATH_INFO: `/v3/projects/bcd37543738f4414be68c94540baa0d3` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:18:07.410525 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada36438-b9c7-4a3e-b05a-937878ff1605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcd37543738f4414be68c94540baa0d3)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:18:07.411591 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: WARNING py.warnings [None req-ada36438-b9c7-4a3e-b05a-937878ff1605 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 12:18:07.411591 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: warnings.warn(msg) Oct 06 12:18:07.411591 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: Oct 06 12:18:07.412798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada36438-b9c7-4a3e-b05a-937878ff1605 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:18:07.469401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ada36438-b9c7-4a3e-b05a-937878ff1605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcd37543738f4414be68c94540baa0d3', 'request_id': 'req-ada36438-b9c7-4a3e-b05a-937878ff1605'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:18:07.478564 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ada36438-b9c7-4a3e-b05a-937878ff1605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcd37543738f4414be68c94540baa0d3', 'request_id': 'req-ada36438-b9c7-4a3e-b05a-937878ff1605'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:18:07.492640 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ada36438-b9c7-4a3e-b05a-937878ff1605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcd37543738f4414be68c94540baa0d3', 'request_id': 'req-ada36438-b9c7-4a3e-b05a-937878ff1605'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:18:07.514935 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ada36438-b9c7-4a3e-b05a-937878ff1605 admin admin] The token cache is being invalidate because project bcd37543738f4414be68c94540baa0d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:18:07.515252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ada36438-b9c7-4a3e-b05a-937878ff1605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:18:07.515718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ada36438-b9c7-4a3e-b05a-937878ff1605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:18:07.517091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 763/1510] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:18:07 2022] DELETE /identity/v3/projects/bcd37543738f4414be68c94540baa0d3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:18:16.930156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=62 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:18:17.512973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:19:59.362050 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2cafc-fba2-4382-815a-8ec87575f650 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:19:59.362631 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2cafc-fba2-4382-815a-8ec87575f650 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:19:59.362631 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2cafc-fba2-4382-815a-8ec87575f650 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:19:59.363975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 12:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:19:59.368825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e952ea7-3df3-4f3a-ab1f-5e177dad342c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:19:59.369260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e952ea7-3df3-4f3a-ab1f-5e177dad342c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:19:59.369364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e952ea7-3df3-4f3a-ab1f-5e177dad342c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:19:59.448301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-0e952ea7-3df3-4f3a-ab1f-5e177dad342c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:19:59.449496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0e952ea7-3df3-4f3a-ab1f-5e177dad342c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:19:59.564929 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 12:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:19:59.716959 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2cafc-fba2-4382-815a-8ec87575f650 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:19:59.719049 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b1f2cafc-fba2-4382-815a-8ec87575f650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:19:59.834373 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2cafc-fba2-4382-815a-8ec87575f650 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:19:59.839732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ed636-f8a7-47d1-88a2-907e4334b3f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:19:59.840336 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ed636-f8a7-47d1-88a2-907e4334b3f3 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:19:59.840613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ed636-f8a7-47d1-88a2-907e4334b3f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:19:59.840885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ed636-f8a7-47d1-88a2-907e4334b3f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:19:59.848738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ed636-f8a7-47d1-88a2-907e4334b3f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:19:59.850295 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ed636-f8a7-47d1-88a2-907e4334b3f3 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:19:59.859332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 749/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:20:00.927352 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723a2a2-fcd7-44f6-a539-e315d6ecb7b9 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:00.928049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723a2a2-fcd7-44f6-a539-e315d6ecb7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:00.928049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723a2a2-fcd7-44f6-a539-e315d6ecb7b9 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:00.929262 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 12:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:20:00.935516 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59344ed-1d7d-4bf5-9bd1-804d8bdd7d34 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:00.935959 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59344ed-1d7d-4bf5-9bd1-804d8bdd7d34 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:00.936080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59344ed-1d7d-4bf5-9bd1-804d8bdd7d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:00.955486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c59344ed-1d7d-4bf5-9bd1-804d8bdd7d34 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:20:00.956702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c59344ed-1d7d-4bf5-9bd1-804d8bdd7d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:20:00.964426 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 12:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:20:01.075182 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae82434-97f1-4c1d-ae2a-54083b0b5bc1 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:01.075597 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae82434-97f1-4c1d-ae2a-54083b0b5bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:01.075962 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae82434-97f1-4c1d-ae2a-54083b0b5bc1 None None] PATH_INFO: `/v3` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:01.077298 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 766/1516] 10.222.0.23 () {54 vars in 925 bytes} [Thu Oct 6 12:20:01 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:20:01.083304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b07077-bdce-4611-b05e-ea35151bfab7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:01.083621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b07077-bdce-4611-b05e-ea35151bfab7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:01.083897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b07077-bdce-4611-b05e-ea35151bfab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:01.119777 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae82434-97f1-4c1d-ae2a-54083b0b5bc1 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:20:01.134484 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae82434-97f1-4c1d-ae2a-54083b0b5bc1 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:20:01.139875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a5713-c908-4e14-9fb6-beb4142ccf2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:20:01.140468 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a5713-c908-4e14-9fb6-beb4142ccf2b service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:01.140744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a5713-c908-4e14-9fb6-beb4142ccf2b service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:01.141016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a5713-c908-4e14-9fb6-beb4142ccf2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:01.143291 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-33b07077-bdce-4611-b05e-ea35151bfab7 None None] MFA Rules not processed for user `39d67f5edf4b441ebaa2bb42e74b2cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:20:01.144520 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-33b07077-bdce-4611-b05e-ea35151bfab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:20:01.149587 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a5713-c908-4e14-9fb6-beb4142ccf2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:20:01.151165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a5713-c908-4e14-9fb6-beb4142ccf2b service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:20:01.161154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 767/1517] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:20:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:20:01.197858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:20:01.218447 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 751/1518] 10.222.0.23 () {58 vars in 1015 bytes} [Thu Oct 6 12:20:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:20:10.233663 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4070b-f74f-473e-a3c3-1cc443ab9fec None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:10.234148 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4070b-f74f-473e-a3c3-1cc443ab9fec None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:10.234259 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4070b-f74f-473e-a3c3-1cc443ab9fec None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:10.235573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 768/1519] 10.222.0.33 () {54 vars in 961 bytes} [Thu Oct 6 12:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:20:10.242114 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cff09-5db3-4e61-bfd6-e9f7a0c227b0 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:10.242608 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cff09-5db3-4e61-bfd6-e9f7a0c227b0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:10.242792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cff09-5db3-4e61-bfd6-e9f7a0c227b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:10.262557 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ab8cff09-5db3-4e61-bfd6-e9f7a0c227b0 None None] MFA Rules not processed for user `39d67f5edf4b441ebaa2bb42e74b2cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:20:10.263832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ab8cff09-5db3-4e61-bfd6-e9f7a0c227b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:20:10.272350 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 752/1520] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Oct 6 12:20:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:20:10.279852 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4070b-f74f-473e-a3c3-1cc443ab9fec None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:20:10.293997 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4070b-f74f-473e-a3c3-1cc443ab9fec None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:20:10.299613 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909f04c-1cbf-4528-b8ae-e3b404f3c357 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:20:10.300204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909f04c-1cbf-4528-b8ae-e3b404f3c357 service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:10.300487 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909f04c-1cbf-4528-b8ae-e3b404f3c357 service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:10.300762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909f04c-1cbf-4528-b8ae-e3b404f3c357 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:10.309041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1909f04c-1cbf-4528-b8ae-e3b404f3c357 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:20:10.310641 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1909f04c-1cbf-4528-b8ae-e3b404f3c357 service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:20:10.320223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 769/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:20:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:20:12.642591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b2603-a083-4391-a52c-e8dc2722b9e7 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:12.643292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b2603-a083-4391-a52c-e8dc2722b9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:12.643401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b2603-a083-4391-a52c-e8dc2722b9e7 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:12.645228 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 12:20:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:20:12.652210 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87a200-d345-490f-b168-34033829ffe1 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:12.652687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87a200-d345-490f-b168-34033829ffe1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:12.652805 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87a200-d345-490f-b168-34033829ffe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:12.672985 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-5f87a200-d345-490f-b168-34033829ffe1 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:20:12.674707 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5f87a200-d345-490f-b168-34033829ffe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:20:12.682696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 12:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:20:12.833483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b2603-a083-4391-a52c-e8dc2722b9e7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:20:12.848731 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b2603-a083-4391-a52c-e8dc2722b9e7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:20:12.854286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9db7c-8eb6-4f1b-ae74-b27eb663d27e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:20:12.854877 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9db7c-8eb6-4f1b-ae74-b27eb663d27e service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:12.855156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9db7c-8eb6-4f1b-ae74-b27eb663d27e service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:12.855427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9db7c-8eb6-4f1b-ae74-b27eb663d27e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:12.863372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9db7c-8eb6-4f1b-ae74-b27eb663d27e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:20:12.865299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9db7c-8eb6-4f1b-ae74-b27eb663d27e service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:20:12.874519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 754/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:20:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:20:13.939910 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a89c3-53ab-4ded-94ea-17870ff548ea None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:13.940481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a89c3-53ab-4ded-94ea-17870ff548ea None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:13.940481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a89c3-53ab-4ded-94ea-17870ff548ea None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:13.941800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 12:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:20:13.947722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28005a1f-79c3-40d2-8d87-eaaa21850584 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:13.948128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28005a1f-79c3-40d2-8d87-eaaa21850584 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:13.948278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28005a1f-79c3-40d2-8d87-eaaa21850584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:13.968138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-28005a1f-79c3-40d2-8d87-eaaa21850584 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:20:13.969332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-28005a1f-79c3-40d2-8d87-eaaa21850584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:20:13.977101 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 12:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:20:14.126846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a89c3-53ab-4ded-94ea-17870ff548ea None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:20:14.140901 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a89c3-53ab-4ded-94ea-17870ff548ea None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:20:14.146786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15696b77-2b5f-44c9-b10c-3112850bbfff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:20:14.147373 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15696b77-2b5f-44c9-b10c-3112850bbfff service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:14.147689 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15696b77-2b5f-44c9-b10c-3112850bbfff service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:14.147964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15696b77-2b5f-44c9-b10c-3112850bbfff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:14.156108 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15696b77-2b5f-44c9-b10c-3112850bbfff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:20:14.157690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15696b77-2b5f-44c9-b10c-3112850bbfff service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:20:14.166846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 772/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:20:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:20:18.239871 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7db8a-6f6a-45dd-b1c2-b6fc1fe71d95 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:18.240439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7db8a-6f6a-45dd-b1c2-b6fc1fe71d95 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:18.240439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7db8a-6f6a-45dd-b1c2-b6fc1fe71d95 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:18.241729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 12:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:20:18.246947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2539d42-3854-4a84-b4f3-6368ddb3b63d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:18.247328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2539d42-3854-4a84-b4f3-6368ddb3b63d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:18.247492 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2539d42-3854-4a84-b4f3-6368ddb3b63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:18.267187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e2539d42-3854-4a84-b4f3-6368ddb3b63d None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:20:18.268406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e2539d42-3854-4a84-b4f3-6368ddb3b63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:20:18.276238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 12:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:20:18.559183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7db8a-6f6a-45dd-b1c2-b6fc1fe71d95 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:20:18.561002 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-34c7db8a-6f6a-45dd-b1c2-b6fc1fe71d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:20:18.637917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7db8a-6f6a-45dd-b1c2-b6fc1fe71d95 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:20:18.641442 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4d3c2-8193-4b1e-b65e-41c40b2caa78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:20:18.642573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4d3c2-8193-4b1e-b65e-41c40b2caa78 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:18.643134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4d3c2-8193-4b1e-b65e-41c40b2caa78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:18.643404 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4d3c2-8193-4b1e-b65e-41c40b2caa78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:18.652230 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4d3c2-8193-4b1e-b65e-41c40b2caa78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:20:18.653704 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4d3c2-8193-4b1e-b65e-41c40b2caa78 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:20:18.663083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 757/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:20:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:20:20.861285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e133db8-e9a6-4797-94ec-293c4d3fcc7c None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:20.861810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e133db8-e9a6-4797-94ec-293c4d3fcc7c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:20.861810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e133db8-e9a6-4797-94ec-293c4d3fcc7c None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:20.863196 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 12:20:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:20:20.868720 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdf720-cb83-4287-a685-993b5369f64b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:20.869141 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdf720-cb83-4287-a685-993b5369f64b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:20.869276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdf720-cb83-4287-a685-993b5369f64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:20.888288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f3fdf720-cb83-4287-a685-993b5369f64b None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:20:20.889517 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f3fdf720-cb83-4287-a685-993b5369f64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:20:20.897243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 12:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:20:21.187697 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e133db8-e9a6-4797-94ec-293c4d3fcc7c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:20:21.198197 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e133db8-e9a6-4797-94ec-293c4d3fcc7c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:20:21.201720 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4078efb-f5d8-4304-aee2-0858f893ed37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:20:21.202304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4078efb-f5d8-4304-aee2-0858f893ed37 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:21.202579 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4078efb-f5d8-4304-aee2-0858f893ed37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:21.202847 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4078efb-f5d8-4304-aee2-0858f893ed37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:21.212044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4078efb-f5d8-4304-aee2-0858f893ed37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:20:21.213451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4078efb-f5d8-4304-aee2-0858f893ed37 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:20:21.223055 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 775/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:20:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:20:25.055093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf341b56-91f1-4275-aa2d-09c5201eb7c1 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:25.055751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf341b56-91f1-4275-aa2d-09c5201eb7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:25.055751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf341b56-91f1-4275-aa2d-09c5201eb7c1 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:25.057031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 12:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:20:25.062198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4218e7b-8d5e-45bf-a163-d01864f958db None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:25.062610 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4218e7b-8d5e-45bf-a163-d01864f958db None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:25.062741 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4218e7b-8d5e-45bf-a163-d01864f958db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:25.070436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:20:25.083839 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f4218e7b-8d5e-45bf-a163-d01864f958db None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:20:25.085036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f4218e7b-8d5e-45bf-a163-d01864f958db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:20:25.092951 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 12:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:20:25.396457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf341b56-91f1-4275-aa2d-09c5201eb7c1 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:20:25.398135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bf341b56-91f1-4275-aa2d-09c5201eb7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:20:25.474220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf341b56-91f1-4275-aa2d-09c5201eb7c1 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:20:25.477728 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f64ed-4637-447c-be5b-f0ffdada6756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:20:25.478323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f64ed-4637-447c-be5b-f0ffdada6756 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:25.478597 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f64ed-4637-447c-be5b-f0ffdada6756 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:25.478863 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f64ed-4637-447c-be5b-f0ffdada6756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:25.487428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f64ed-4637-447c-be5b-f0ffdada6756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:20:25.488861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f64ed-4637-447c-be5b-f0ffdada6756 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:20:25.497897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 760/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:20:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:20:27.613582 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65ba4d-d508-477c-a32a-b08c136bad87 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:27.614063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65ba4d-d508-477c-a32a-b08c136bad87 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:27.614194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65ba4d-d508-477c-a32a-b08c136bad87 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:27.615467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 12:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:20:27.620708 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aca16e-7585-4ebb-af14-1c5d8f0e0b63 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:27.621120 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aca16e-7585-4ebb-af14-1c5d8f0e0b63 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:27.621274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aca16e-7585-4ebb-af14-1c5d8f0e0b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:27.641038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-32aca16e-7585-4ebb-af14-1c5d8f0e0b63 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:20:27.642816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-32aca16e-7585-4ebb-af14-1c5d8f0e0b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:20:27.650439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 12:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:20:27.957958 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65ba4d-d508-477c-a32a-b08c136bad87 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:20:27.959654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cd65ba4d-d508-477c-a32a-b08c136bad87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:20:27.972825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65ba4d-d508-477c-a32a-b08c136bad87 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:20:27.976298 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c502c1-785b-4c9f-8aa7-b3ac216814f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:20:27.976881 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c502c1-785b-4c9f-8aa7-b3ac216814f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:20:27.977157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c502c1-785b-4c9f-8aa7-b3ac216814f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:20:27.977430 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c502c1-785b-4c9f-8aa7-b3ac216814f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:20:27.986041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c502c1-785b-4c9f-8aa7-b3ac216814f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:20:27.987442 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c502c1-785b-4c9f-8aa7-b3ac216814f7 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:20:27.996540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 778/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:20:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:20:37.636343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:20:37.992494 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=8 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:07.865308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10cbed-83f8-4c21-87b8-6669531277f3 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:07.866495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10cbed-83f8-4c21-87b8-6669531277f3 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:07.867012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10cbed-83f8-4c21-87b8-6669531277f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:07.892830 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2a10cbed-83f8-4c21-87b8-6669531277f3 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:07.894046 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2a10cbed-83f8-4c21-87b8-6669531277f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:07.908165 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 762/1540] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:07.917429 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb33ad-26d7-432c-9631-d9cca483017b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:07.918029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb33ad-26d7-432c-9631-d9cca483017b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:07.918029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb33ad-26d7-432c-9631-d9cca483017b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:07.948867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f1eb33ad-26d7-432c-9631-d9cca483017b None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:07.950147 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f1eb33ad-26d7-432c-9631-d9cca483017b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:07.968322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 779/1541] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:07.972702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10cbed-83f8-4c21-87b8-6669531277f3 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:07.984288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10cbed-83f8-4c21-87b8-6669531277f3 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:07.992016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db786e7-7269-4274-b237-021291156962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:07.992657 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db786e7-7269-4274-b237-021291156962 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:07.992935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db786e7-7269-4274-b237-021291156962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:07.993205 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db786e7-7269-4274-b237-021291156962 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:07.993862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db786e7-7269-4274-b237-021291156962 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:07.994163 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db786e7-7269-4274-b237-021291156962 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:07.995583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: WARNING py.warnings [None req-0db786e7-7269-4274-b237-021291156962 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 06 12:22:07.995583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: warnings.warn(msg) Oct 06 12:22:07.995583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: Oct 06 12:22:07.996472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db786e7-7269-4274-b237-021291156962 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.017092 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 763/1542] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:08.025485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb33ad-26d7-432c-9631-d9cca483017b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.040724 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb33ad-26d7-432c-9631-d9cca483017b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.048724 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64d0f5-4be0-4349-a4ba-622f5eada9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:08.049472 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64d0f5-4be0-4349-a4ba-622f5eada9dc admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.049759 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64d0f5-4be0-4349-a4ba-622f5eada9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.050033 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64d0f5-4be0-4349-a4ba-622f5eada9dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.050671 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64d0f5-4be0-4349-a4ba-622f5eada9dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:08.051845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: WARNING py.warnings [None req-dd64d0f5-4be0-4349-a4ba-622f5eada9dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 12:22:08.051845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: warnings.warn(msg) Oct 06 12:22:08.051845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: Oct 06 12:22:08.053039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64d0f5-4be0-4349-a4ba-622f5eada9dc admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.110889 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48396b92-eb45-4100-9cbe-9a617eae2d2a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.111645 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48396b92-eb45-4100-9cbe-9a617eae2d2a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.111645 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48396b92-eb45-4100-9cbe-9a617eae2d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.121477 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 780/1543] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/projects => generated 391 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:08.123304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373954a-359b-48ea-9ced-eb7d0c40eba1 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.123637 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373954a-359b-48ea-9ced-eb7d0c40eba1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.123916 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373954a-359b-48ea-9ced-eb7d0c40eba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.142941 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-48396b92-eb45-4100-9cbe-9a617eae2d2a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:08.144171 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-48396b92-eb45-4100-9cbe-9a617eae2d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:08.164489 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-a373954a-359b-48ea-9ced-eb7d0c40eba1 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:08.166206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a373954a-359b-48ea-9ced-eb7d0c40eba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:08.260464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 764/1544] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:22:08.261396 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48396b92-eb45-4100-9cbe-9a617eae2d2a None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.269200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 781/1545] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:08.270740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90410f-1629-4221-ab29-a0f334bb59ef None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.271232 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90410f-1629-4221-ab29-a0f334bb59ef None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.271357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48396b92-eb45-4100-9cbe-9a617eae2d2a None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.271971 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90410f-1629-4221-ab29-a0f334bb59ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.276449 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f66c0-1d49-42dc-b6a4-fe4ad616f84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:08.277065 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f66c0-1d49-42dc-b6a4-fe4ad616f84b admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.277342 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f66c0-1d49-42dc-b6a4-fe4ad616f84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.277752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f66c0-1d49-42dc-b6a4-fe4ad616f84b admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.278402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f66c0-1d49-42dc-b6a4-fe4ad616f84b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:08.279457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: WARNING py.warnings [None req-1e0f66c0-1d49-42dc-b6a4-fe4ad616f84b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 12:22:08.279457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: warnings.warn(msg) Oct 06 12:22:08.279457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: Oct 06 12:22:08.280947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f66c0-1d49-42dc-b6a4-fe4ad616f84b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.293585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-aa90410f-1629-4221-ab29-a0f334bb59ef None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:08.295481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-aa90410f-1629-4221-ab29-a0f334bb59ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:08.304158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 782/1546] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:08.305351 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33431dcb-ba52-4a54-83dd-c1b75b3b5537 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.305634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33431dcb-ba52-4a54-83dd-c1b75b3b5537 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.305904 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33431dcb-ba52-4a54-83dd-c1b75b3b5537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.306518 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 765/1547] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/users => generated 296 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:08.308114 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc1798-c1e2-40d8-bb74-9d267f627073 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.308547 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc1798-c1e2-40d8-bb74-9d267f627073 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.308828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc1798-c1e2-40d8-bb74-9d267f627073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.328802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-33431dcb-ba52-4a54-83dd-c1b75b3b5537 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:08.329018 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-0bfc1798-c1e2-40d8-bb74-9d267f627073 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:08.330040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-33431dcb-ba52-4a54-83dd-c1b75b3b5537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:08.330166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0bfc1798-c1e2-40d8-bb74-9d267f627073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:08.338405 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 766/1548] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:08.339005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 783/1549] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:08.340101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c0b96-7e17-4310-9eb8-17cce159af86 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.340380 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c0b96-7e17-4310-9eb8-17cce159af86 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.340651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c0b96-7e17-4310-9eb8-17cce159af86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.344363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4bee3-8f73-4df6-be59-7c4e90189aed None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.344652 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4bee3-8f73-4df6-be59-7c4e90189aed None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.344924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4bee3-8f73-4df6-be59-7c4e90189aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.364524 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-189c0b96-7e17-4310-9eb8-17cce159af86 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:08.364722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ebc4bee3-8f73-4df6-be59-7c4e90189aed None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:08.365725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-189c0b96-7e17-4310-9eb8-17cce159af86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:08.365901 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ebc4bee3-8f73-4df6-be59-7c4e90189aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:08.374418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 784/1550] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:08.374575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 767/1551] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:08.375623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b4378-ba9f-473c-a215-8b8caeaf6052 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.375817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcdabb-c095-4cbf-9427-966605971a19 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.375922 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b4378-ba9f-473c-a215-8b8caeaf6052 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.376092 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcdabb-c095-4cbf-9427-966605971a19 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.376205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b4378-ba9f-473c-a215-8b8caeaf6052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.376371 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcdabb-c095-4cbf-9427-966605971a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.396803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-519b4378-ba9f-473c-a215-8b8caeaf6052 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:08.397024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9fdcdabb-c095-4cbf-9427-966605971a19 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:08.397919 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-519b4378-ba9f-473c-a215-8b8caeaf6052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:08.398206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9fdcdabb-c095-4cbf-9427-966605971a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:08.406205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 785/1552] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:08.411948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 768/1553] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:08.412098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b4378-ba9f-473c-a215-8b8caeaf6052 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.412212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcacdc-12e2-40b9-a974-16008f0417a9 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.412212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcacdc-12e2-40b9-a974-16008f0417a9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.412212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcacdc-12e2-40b9-a974-16008f0417a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.418938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b4378-ba9f-473c-a215-8b8caeaf6052 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.424134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8562c-c741-464f-b98f-f3f751cc0c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:08.424735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8562c-c741-464f-b98f-f3f751cc0c0e admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.425042 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8562c-c741-464f-b98f-f3f751cc0c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.425320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8562c-c741-464f-b98f-f3f751cc0c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.425959 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8562c-c741-464f-b98f-f3f751cc0c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:08.427696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8562c-c741-464f-b98f-f3f751cc0c0e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.428734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-87fcacdc-12e2-40b9-a974-16008f0417a9 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:08.429918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-87fcacdc-12e2-40b9-a974-16008f0417a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:08.438367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 769/1554] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:08.441799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 786/1555] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:08.441935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419fd4a-b669-45cf-a54d-b07bbe8f9843 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.441935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419fd4a-b669-45cf-a54d-b07bbe8f9843 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.442150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558da67-0312-44d8-833c-8a8032f56129 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.442277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419fd4a-b669-45cf-a54d-b07bbe8f9843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.442377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558da67-0312-44d8-833c-8a8032f56129 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.442377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558da67-0312-44d8-833c-8a8032f56129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.460249 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f419fd4a-b669-45cf-a54d-b07bbe8f9843 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:08.460618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-0558da67-0312-44d8-833c-8a8032f56129 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:08.461431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f419fd4a-b669-45cf-a54d-b07bbe8f9843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:08.461792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0558da67-0312-44d8-833c-8a8032f56129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:08.469356 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 770/1556] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:08.469903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 787/1557] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:08.470121 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419fd4a-b669-45cf-a54d-b07bbe8f9843 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.471145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882ab10-9423-4c60-ba68-e5f5e4f09fce None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.471430 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882ab10-9423-4c60-ba68-e5f5e4f09fce None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.471741 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882ab10-9423-4c60-ba68-e5f5e4f09fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.479714 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419fd4a-b669-45cf-a54d-b07bbe8f9843 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.485407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e2ca5-9e86-4ca8-88e9-41fd72742b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:08.486002 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e2ca5-9e86-4ca8-88e9-41fd72742b25 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.486279 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e2ca5-9e86-4ca8-88e9-41fd72742b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.486553 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e2ca5-9e86-4ca8-88e9-41fd72742b25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.487186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e2ca5-9e86-4ca8-88e9-41fd72742b25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:08.487488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e2ca5-9e86-4ca8-88e9-41fd72742b25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:08.489106 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e2ca5-9e86-4ca8-88e9-41fd72742b25 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.492323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3882ab10-9423-4c60-ba68-e5f5e4f09fce None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:08.493523 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3882ab10-9423-4c60-ba68-e5f5e4f09fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:08.508579 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 771/1558] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:08.508579 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e2ca5-9e86-4ca8-88e9-41fd72742b25 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.508885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 788/1559] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:08.508885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7cebf-cb14-4b97-ae3d-2b7fbefa12ca None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.508885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7cebf-cb14-4b97-ae3d-2b7fbefa12ca None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.508885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7cebf-cb14-4b97-ae3d-2b7fbefa12ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.515028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e2ca5-9e86-4ca8-88e9-41fd72742b25 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.519649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2430720-c02a-4653-b5d7-c6453033b602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:08.520246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2430720-c02a-4653-b5d7-c6453033b602 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.520528 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2430720-c02a-4653-b5d7-c6453033b602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.520805 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2430720-c02a-4653-b5d7-c6453033b602 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.521435 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2430720-c02a-4653-b5d7-c6453033b602 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:08.521738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2430720-c02a-4653-b5d7-c6453033b602 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:08.523174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c3d7cebf-cb14-4b97-ae3d-2b7fbefa12ca None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:08.523303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2430720-c02a-4653-b5d7-c6453033b602 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.524472 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c3d7cebf-cb14-4b97-ae3d-2b7fbefa12ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:08.533232 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 789/1560] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:08.534009 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7cebf-cb14-4b97-ae3d-2b7fbefa12ca None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.539986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 772/1561] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:08.540409 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2430720-c02a-4653-b5d7-c6453033b602 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.543786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7cebf-cb14-4b97-ae3d-2b7fbefa12ca None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.548964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb04a72-4d88-45b7-a3e0-1b92a500ab05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:08.549568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb04a72-4d88-45b7-a3e0-1b92a500ab05 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.549853 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb04a72-4d88-45b7-a3e0-1b92a500ab05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.550129 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb04a72-4d88-45b7-a3e0-1b92a500ab05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.550799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb04a72-4d88-45b7-a3e0-1b92a500ab05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:08.551466 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2430720-c02a-4653-b5d7-c6453033b602 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.552425 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb04a72-4d88-45b7-a3e0-1b92a500ab05 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.556293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad986caa-fa59-41a9-b306-252776b34338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:08.556953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad986caa-fa59-41a9-b306-252776b34338 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.557237 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad986caa-fa59-41a9-b306-252776b34338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.557510 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad986caa-fa59-41a9-b306-252776b34338 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.558469 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad986caa-fa59-41a9-b306-252776b34338 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:08.558469 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad986caa-fa59-41a9-b306-252776b34338 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:08.565997 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 790/1562] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:08.565997 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb04a72-4d88-45b7-a3e0-1b92a500ab05 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.568920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad986caa-fa59-41a9-b306-252776b34338 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.574635 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb04a72-4d88-45b7-a3e0-1b92a500ab05 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.580377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de99ec-0bf4-49d8-ab33-24af247c4ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:08.581385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de99ec-0bf4-49d8-ab33-24af247c4ffe admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.581763 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de99ec-0bf4-49d8-ab33-24af247c4ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.589342 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 773/1563] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:08.589526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de99ec-0bf4-49d8-ab33-24af247c4ffe admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.589526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de99ec-0bf4-49d8-ab33-24af247c4ffe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:08.589722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad986caa-fa59-41a9-b306-252776b34338 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.589818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de99ec-0bf4-49d8-ab33-24af247c4ffe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:08.589818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de99ec-0bf4-49d8-ab33-24af247c4ffe admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.595285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad986caa-fa59-41a9-b306-252776b34338 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.606752 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 791/1564] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:08.606752 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de99ec-0bf4-49d8-ab33-24af247c4ffe admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.607088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be73af-61d0-4229-86bb-4153a1190999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:08.607088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be73af-61d0-4229-86bb-4153a1190999 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.607088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be73af-61d0-4229-86bb-4153a1190999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.607088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be73af-61d0-4229-86bb-4153a1190999 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.607496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be73af-61d0-4229-86bb-4153a1190999 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:08.607496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be73af-61d0-4229-86bb-4153a1190999 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:08.607496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be73af-61d0-4229-86bb-4153a1190999 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.611846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de99ec-0bf4-49d8-ab33-24af247c4ffe admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.616603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0df08-ff1d-482a-a0ca-2c6e9261fa37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:08.617219 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0df08-ff1d-482a-a0ca-2c6e9261fa37 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.617494 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0df08-ff1d-482a-a0ca-2c6e9261fa37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.617764 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0df08-ff1d-482a-a0ca-2c6e9261fa37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.618416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0df08-ff1d-482a-a0ca-2c6e9261fa37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:08.618721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0df08-ff1d-482a-a0ca-2c6e9261fa37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:08.619245 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 774/1565] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:08.619245 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be73af-61d0-4229-86bb-4153a1190999 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.627387 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0df08-ff1d-482a-a0ca-2c6e9261fa37 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.631335 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be73af-61d0-4229-86bb-4153a1190999 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.636679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98350d-2587-4dd4-9dfa-359eb408afc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:08.637433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98350d-2587-4dd4-9dfa-359eb408afc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.637788 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98350d-2587-4dd4-9dfa-359eb408afc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.638436 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98350d-2587-4dd4-9dfa-359eb408afc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.639078 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 792/1566] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:08.639522 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98350d-2587-4dd4-9dfa-359eb408afc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:08.641543 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98350d-2587-4dd4-9dfa-359eb408afc9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.644901 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0df08-ff1d-482a-a0ca-2c6e9261fa37 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.658970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0df08-ff1d-482a-a0ca-2c6e9261fa37 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.665070 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bef4e1-0c32-426e-922c-ada9f10f07aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:08.666543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bef4e1-0c32-426e-922c-ada9f10f07aa admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.667040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bef4e1-0c32-426e-922c-ada9f10f07aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.667560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bef4e1-0c32-426e-922c-ada9f10f07aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.668688 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bef4e1-0c32-426e-922c-ada9f10f07aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:08.669187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bef4e1-0c32-426e-922c-ada9f10f07aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:08.671435 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bef4e1-0c32-426e-922c-ada9f10f07aa admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.672765 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 775/1567] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:08.673579 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98350d-2587-4dd4-9dfa-359eb408afc9 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.686304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 793/1568] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:08.688174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bef4e1-0c32-426e-922c-ada9f10f07aa admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.695334 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:08.705075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bef4e1-0c32-426e-922c-ada9f10f07aa admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.719089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:08.749045 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98350d-2587-4dd4-9dfa-359eb408afc9 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.755952 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74025a6f-5796-4247-ad7d-0ad12cbe2986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:08.756575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74025a6f-5796-4247-ad7d-0ad12cbe2986 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.756861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74025a6f-5796-4247-ad7d-0ad12cbe2986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.757343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74025a6f-5796-4247-ad7d-0ad12cbe2986 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.757986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74025a6f-5796-4247-ad7d-0ad12cbe2986 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:08.759940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74025a6f-5796-4247-ad7d-0ad12cbe2986 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.770176 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7f4d2-dc19-45d6-8d2f-7e1b198c1e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:08.770840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7f4d2-dc19-45d6-8d2f-7e1b198c1e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.771133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7f4d2-dc19-45d6-8d2f-7e1b198c1e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.771409 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7f4d2-dc19-45d6-8d2f-7e1b198c1e22 admin admin] PATH_INFO: `/v3/projects/9138bc89f5204975a18156097530643a/users/1f4cbe342b2c4378b3c0ab2f1cd55260/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.787809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7f4d2-dc19-45d6-8d2f-7e1b198c1e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9138bc89f5204975a18156097530643a, user_id=1f4cbe342b2c4378b3c0ab2f1cd55260, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:08.788873 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: WARNING py.warnings [None req-74b7f4d2-dc19-45d6-8d2f-7e1b198c1e22 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 12:22:08.788873 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: warnings.warn(msg) Oct 06 12:22:08.788873 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: Oct 06 12:22:08.790040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 776/1569] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/projects => generated 403 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:08.790798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7f4d2-dc19-45d6-8d2f-7e1b198c1e22 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.790908 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74025a6f-5796-4247-ad7d-0ad12cbe2986 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.804283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 794/1570] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:08 2022] PUT /identity/v3/projects/9138bc89f5204975a18156097530643a/users/1f4cbe342b2c4378b3c0ab2f1cd55260/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:08.808276 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7f4d2-dc19-45d6-8d2f-7e1b198c1e22 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.852849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74025a6f-5796-4247-ad7d-0ad12cbe2986 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.865289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7f4d2-dc19-45d6-8d2f-7e1b198c1e22 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.869858 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70037a41-3db9-4581-ad3b-ba615a76efb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:08.870785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70037a41-3db9-4581-ad3b-ba615a76efb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.870785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70037a41-3db9-4581-ad3b-ba615a76efb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.871019 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70037a41-3db9-4581-ad3b-ba615a76efb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.871616 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70037a41-3db9-4581-ad3b-ba615a76efb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:08.874021 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70037a41-3db9-4581-ad3b-ba615a76efb1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.905557 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 795/1571] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:08.905557 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70037a41-3db9-4581-ad3b-ba615a76efb1 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.909888 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67fc00-7a32-44cb-9a34-69cc18faf28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:08.910445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67fc00-7a32-44cb-9a34-69cc18faf28d admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:08.910716 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67fc00-7a32-44cb-9a34-69cc18faf28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:08.910993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67fc00-7a32-44cb-9a34-69cc18faf28d admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:08.911661 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67fc00-7a32-44cb-9a34-69cc18faf28d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:08.913594 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67fc00-7a32-44cb-9a34-69cc18faf28d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:08.944817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 777/1572] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/projects => generated 365 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:08.944817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67fc00-7a32-44cb-9a34-69cc18faf28d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:08.968730 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70037a41-3db9-4581-ad3b-ba615a76efb1 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:08.977674 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:08.999120 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:09.007087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67fc00-7a32-44cb-9a34-69cc18faf28d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.011729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989ec73-1606-48a2-8375-125251670c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.012332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989ec73-1606-48a2-8375-125251670c61 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.012609 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989ec73-1606-48a2-8375-125251670c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.012873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989ec73-1606-48a2-8375-125251670c61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.014044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989ec73-1606-48a2-8375-125251670c61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.017849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989ec73-1606-48a2-8375-125251670c61 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.030566 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86ad18-5088-4a0a-a458-c41349eb8b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.031174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86ad18-5088-4a0a-a458-c41349eb8b74 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.031451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86ad18-5088-4a0a-a458-c41349eb8b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.031843 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86ad18-5088-4a0a-a458-c41349eb8b74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.032471 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86ad18-5088-4a0a-a458-c41349eb8b74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.034387 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86ad18-5088-4a0a-a458-c41349eb8b74 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.045898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 778/1573] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:09.049887 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989ec73-1606-48a2-8375-125251670c61 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.062142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 796/1574] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:08 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:09.063145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86ad18-5088-4a0a-a458-c41349eb8b74 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.112038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989ec73-1606-48a2-8375-125251670c61 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.126503 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86ad18-5088-4a0a-a458-c41349eb8b74 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.131285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85fc15-5656-4c07-b999-019e06fb3cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.131945 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85fc15-5656-4c07-b999-019e06fb3cfe admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.132247 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85fc15-5656-4c07-b999-019e06fb3cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.132505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85fc15-5656-4c07-b999-019e06fb3cfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.133134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85fc15-5656-4c07-b999-019e06fb3cfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.135067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85fc15-5656-4c07-b999-019e06fb3cfe admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.162778 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 797/1575] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:09 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:09.163619 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85fc15-5656-4c07-b999-019e06fb3cfe admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.166762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667da6f-f89d-42fa-9334-a57292124563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.167387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667da6f-f89d-42fa-9334-a57292124563 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.167707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667da6f-f89d-42fa-9334-a57292124563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.168044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667da6f-f89d-42fa-9334-a57292124563 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.168715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667da6f-f89d-42fa-9334-a57292124563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.171195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667da6f-f89d-42fa-9334-a57292124563 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.194054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 779/1576] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:09 2022] POST /identity/v3/users => generated 286 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:09.195295 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287b836-493b-4bd9-ae16-29f227408b30 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.195587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287b836-493b-4bd9-ae16-29f227408b30 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.195858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287b836-493b-4bd9-ae16-29f227408b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.214368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d287b836-493b-4bd9-ae16-29f227408b30 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:09.215597 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d287b836-493b-4bd9-ae16-29f227408b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:09.226358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85fc15-5656-4c07-b999-019e06fb3cfe admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.230998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a54fe-3109-423c-b321-afc3531f9e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.231745 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a54fe-3109-423c-b321-afc3531f9e3d admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.231965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a54fe-3109-423c-b321-afc3531f9e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.232793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a54fe-3109-423c-b321-afc3531f9e3d admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.233569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a54fe-3109-423c-b321-afc3531f9e3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.235482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a54fe-3109-423c-b321-afc3531f9e3d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.261594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 798/1577] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:09 2022] POST /identity/v3/users => generated 302 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:09.261594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a54fe-3109-423c-b321-afc3531f9e3d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.268356 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:09.269035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a54fe-3109-423c-b321-afc3531f9e3d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.274012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3953a8-c7d3-4a43-baf4-7cf3265beb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.274602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3953a8-c7d3-4a43-baf4-7cf3265beb27 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.274969 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3953a8-c7d3-4a43-baf4-7cf3265beb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.275334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3953a8-c7d3-4a43-baf4-7cf3265beb27 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.276097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3953a8-c7d3-4a43-baf4-7cf3265beb27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.278017 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3953a8-c7d3-4a43-baf4-7cf3265beb27 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.281167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 780/1578] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:09.281933 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287b836-493b-4bd9-ae16-29f227408b30 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.291838 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287b836-493b-4bd9-ae16-29f227408b30 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.297281 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f8751-002c-4769-adec-3dd05388a802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.297863 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f8751-002c-4769-adec-3dd05388a802 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.298145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f8751-002c-4769-adec-3dd05388a802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.298415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f8751-002c-4769-adec-3dd05388a802 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.299039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f8751-002c-4769-adec-3dd05388a802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.300539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 799/1579] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:09 2022] POST /identity/v3/users => generated 289 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:09.300995 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f8751-002c-4769-adec-3dd05388a802 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.301345 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3953a8-c7d3-4a43-baf4-7cf3265beb27 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.312070 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3953a8-c7d3-4a43-baf4-7cf3265beb27 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.316829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33551c1-7291-4e74-9d50-6d61d4ecff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.317450 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33551c1-7291-4e74-9d50-6d61d4ecff62 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.317736 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33551c1-7291-4e74-9d50-6d61d4ecff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.318030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33551c1-7291-4e74-9d50-6d61d4ecff62 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.318644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33551c1-7291-4e74-9d50-6d61d4ecff62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.320581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33551c1-7291-4e74-9d50-6d61d4ecff62 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.325087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 781/1580] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:09 2022] POST /identity/v3/users => generated 283 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:09.325087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f8751-002c-4769-adec-3dd05388a802 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.332650 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f8751-002c-4769-adec-3dd05388a802 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.335690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:09.338522 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85bf03-4b24-4ee9-88ca-d32334491c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.339234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85bf03-4b24-4ee9-88ca-d32334491c3d admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.339603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85bf03-4b24-4ee9-88ca-d32334491c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.339916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85bf03-4b24-4ee9-88ca-d32334491c3d admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.340619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85bf03-4b24-4ee9-88ca-d32334491c3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.343643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85bf03-4b24-4ee9-88ca-d32334491c3d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.348675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 800/1581] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:09 2022] POST /identity/v3/users => generated 282 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:09.348675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33551c1-7291-4e74-9d50-6d61d4ecff62 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.361423 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33551c1-7291-4e74-9d50-6d61d4ecff62 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.364930 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 782/1582] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:09 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:09.365583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85bf03-4b24-4ee9-88ca-d32334491c3d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.366483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d56eb7-167a-49ff-9a91-b4cadeb09bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.367221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d56eb7-167a-49ff-9a91-b4cadeb09bba admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.367542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d56eb7-167a-49ff-9a91-b4cadeb09bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.368089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d56eb7-167a-49ff-9a91-b4cadeb09bba admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.368724 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d56eb7-167a-49ff-9a91-b4cadeb09bba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.370632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d56eb7-167a-49ff-9a91-b4cadeb09bba admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.375456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85bf03-4b24-4ee9-88ca-d32334491c3d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.380006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414d6ad-d10d-42ac-a15b-b2d285515b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.380583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414d6ad-d10d-42ac-a15b-b2d285515b92 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.380860 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414d6ad-d10d-42ac-a15b-b2d285515b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.381129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414d6ad-d10d-42ac-a15b-b2d285515b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.381781 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414d6ad-d10d-42ac-a15b-b2d285515b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.383347 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414d6ad-d10d-42ac-a15b-b2d285515b92 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.396086 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 801/1583] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:09 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:09.396086 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d56eb7-167a-49ff-9a91-b4cadeb09bba admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.396353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 783/1584] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:09.396353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414d6ad-d10d-42ac-a15b-b2d285515b92 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.397208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5414d6ad-d10d-42ac-a15b-b2d285515b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:09.406500 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d56eb7-167a-49ff-9a91-b4cadeb09bba admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.411318 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d20cf-a5a6-42c5-b3a1-7648fda95344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.411949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d20cf-a5a6-42c5-b3a1-7648fda95344 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.412261 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d20cf-a5a6-42c5-b3a1-7648fda95344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.412530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d20cf-a5a6-42c5-b3a1-7648fda95344 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.413218 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d20cf-a5a6-42c5-b3a1-7648fda95344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.414763 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d20cf-a5a6-42c5-b3a1-7648fda95344 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.424743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 802/1585] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:09.425601 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d20cf-a5a6-42c5-b3a1-7648fda95344 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.437120 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d20cf-a5a6-42c5-b3a1-7648fda95344 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.442252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f69e58-5628-4e4c-a36d-9678643163d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.442903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f69e58-5628-4e4c-a36d-9678643163d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.443313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f69e58-5628-4e4c-a36d-9678643163d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.443627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f69e58-5628-4e4c-a36d-9678643163d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.444273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f69e58-5628-4e4c-a36d-9678643163d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.445867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f69e58-5628-4e4c-a36d-9678643163d5 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.451935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414d6ad-d10d-42ac-a15b-b2d285515b92 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.455606 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbc114-17d8-4837-bf16-807d872fa220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.456234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbc114-17d8-4837-bf16-807d872fa220 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.456458 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 803/1586] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:09.459639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbc114-17d8-4837-bf16-807d872fa220 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.459639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbc114-17d8-4837-bf16-807d872fa220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.459824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f69e58-5628-4e4c-a36d-9678643163d5 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.467386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f69e58-5628-4e4c-a36d-9678643163d5 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.468631 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbc114-17d8-4837-bf16-807d872fa220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.470059 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbc114-17d8-4837-bf16-807d872fa220 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.472256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e433f-c0d1-463e-8cd8-d81a3560c6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.472822 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e433f-c0d1-463e-8cd8-d81a3560c6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.473102 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e433f-c0d1-463e-8cd8-d81a3560c6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.473375 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e433f-c0d1-463e-8cd8-d81a3560c6e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.473995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e433f-c0d1-463e-8cd8-d81a3560c6e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.475571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e433f-c0d1-463e-8cd8-d81a3560c6e6 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.479576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 784/1587] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:09.480357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbc114-17d8-4837-bf16-807d872fa220 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.488390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 804/1588] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:09.488390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e433f-c0d1-463e-8cd8-d81a3560c6e6 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.489434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbc114-17d8-4837-bf16-807d872fa220 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.495079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b204b8-a17c-4a15-9fcf-e120111b3fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.495703 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b204b8-a17c-4a15-9fcf-e120111b3fbb admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.495984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b204b8-a17c-4a15-9fcf-e120111b3fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.496128 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e433f-c0d1-463e-8cd8-d81a3560c6e6 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.496271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b204b8-a17c-4a15-9fcf-e120111b3fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.497266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b204b8-a17c-4a15-9fcf-e120111b3fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.499143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b204b8-a17c-4a15-9fcf-e120111b3fbb admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.502400 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82a925-8726-404e-a82c-22b21a2f7690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.502925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82a925-8726-404e-a82c-22b21a2f7690 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.503205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82a925-8726-404e-a82c-22b21a2f7690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.503481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82a925-8726-404e-a82c-22b21a2f7690 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.504220 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82a925-8726-404e-a82c-22b21a2f7690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.506252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82a925-8726-404e-a82c-22b21a2f7690 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.510797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 785/1589] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:09.511499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b204b8-a17c-4a15-9fcf-e120111b3fbb admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.518309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 805/1590] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:09.519267 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82a925-8726-404e-a82c-22b21a2f7690 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.525024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b204b8-a17c-4a15-9fcf-e120111b3fbb admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.530054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82a925-8726-404e-a82c-22b21a2f7690 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.530898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e68e9b-1030-4b76-ae5b-08e1eac480bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.531542 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e68e9b-1030-4b76-ae5b-08e1eac480bf admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.531815 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e68e9b-1030-4b76-ae5b-08e1eac480bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.532125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e68e9b-1030-4b76-ae5b-08e1eac480bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.532715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e68e9b-1030-4b76-ae5b-08e1eac480bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.534445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e68e9b-1030-4b76-ae5b-08e1eac480bf admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.535659 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685035c0-cca3-4c91-80a4-d8e5fe5d9a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.536273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685035c0-cca3-4c91-80a4-d8e5fe5d9a11 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.536588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685035c0-cca3-4c91-80a4-d8e5fe5d9a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.536862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685035c0-cca3-4c91-80a4-d8e5fe5d9a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.537497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685035c0-cca3-4c91-80a4-d8e5fe5d9a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.539140 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685035c0-cca3-4c91-80a4-d8e5fe5d9a11 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.545111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 786/1591] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:09.545794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e68e9b-1030-4b76-ae5b-08e1eac480bf admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.556013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e68e9b-1030-4b76-ae5b-08e1eac480bf admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.556214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 806/1592] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:09.557020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685035c0-cca3-4c91-80a4-d8e5fe5d9a11 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.561537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a56e6a-d25a-4a87-b441-3d3b1b4d745e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.562275 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a56e6a-d25a-4a87-b441-3d3b1b4d745e admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.562769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a56e6a-d25a-4a87-b441-3d3b1b4d745e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.563112 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a56e6a-d25a-4a87-b441-3d3b1b4d745e admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.564369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a56e6a-d25a-4a87-b441-3d3b1b4d745e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.566159 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a56e6a-d25a-4a87-b441-3d3b1b4d745e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.569227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685035c0-cca3-4c91-80a4-d8e5fe5d9a11 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.575561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670cfb3-2d68-46aa-a4ac-2ce987885074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.576563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670cfb3-2d68-46aa-a4ac-2ce987885074 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.576888 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670cfb3-2d68-46aa-a4ac-2ce987885074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.577195 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670cfb3-2d68-46aa-a4ac-2ce987885074 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.577872 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f670cfb3-2d68-46aa-a4ac-2ce987885074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.578305 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 787/1593] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:09.579140 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a56e6a-d25a-4a87-b441-3d3b1b4d745e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.579594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f670cfb3-2d68-46aa-a4ac-2ce987885074 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.591312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a56e6a-d25a-4a87-b441-3d3b1b4d745e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.592422 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 807/1594] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:09.601881 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670cfb3-2d68-46aa-a4ac-2ce987885074 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.602271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f01f5-e7db-49e2-aebc-1776ef2474ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.603062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f01f5-e7db-49e2-aebc-1776ef2474ea admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.603406 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f01f5-e7db-49e2-aebc-1776ef2474ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.603800 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f01f5-e7db-49e2-aebc-1776ef2474ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.604447 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f01f5-e7db-49e2-aebc-1776ef2474ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.606067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f01f5-e7db-49e2-aebc-1776ef2474ea admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.616113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670cfb3-2d68-46aa-a4ac-2ce987885074 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.620972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 788/1595] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:09.620972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f01f5-e7db-49e2-aebc-1776ef2474ea admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.623583 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7fd29-3876-4c06-a3d1-e643540ab450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.624474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7fd29-3876-4c06-a3d1-e643540ab450 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.624836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7fd29-3876-4c06-a3d1-e643540ab450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.625183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7fd29-3876-4c06-a3d1-e643540ab450 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.626514 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7fd29-3876-4c06-a3d1-e643540ab450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.628585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7fd29-3876-4c06-a3d1-e643540ab450 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.630316 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f01f5-e7db-49e2-aebc-1776ef2474ea admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.635962 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35eef27-d899-4ee6-a452-7b7941d147ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.636771 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35eef27-d899-4ee6-a452-7b7941d147ce admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.636888 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35eef27-d899-4ee6-a452-7b7941d147ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.637222 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35eef27-d899-4ee6-a452-7b7941d147ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.637774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35eef27-d899-4ee6-a452-7b7941d147ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.640129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35eef27-d899-4ee6-a452-7b7941d147ce admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.645312 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 808/1596] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:09.646315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7fd29-3876-4c06-a3d1-e643540ab450 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.654354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 789/1597] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:09.654354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35eef27-d899-4ee6-a452-7b7941d147ce admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.658279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7fd29-3876-4c06-a3d1-e643540ab450 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.662427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35eef27-d899-4ee6-a452-7b7941d147ce admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.664639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7f5b9-b94b-4569-a7b8-c82d84be948b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.665224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7f5b9-b94b-4569-a7b8-c82d84be948b admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.665520 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7f5b9-b94b-4569-a7b8-c82d84be948b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.665789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7f5b9-b94b-4569-a7b8-c82d84be948b admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.666442 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7f5b9-b94b-4569-a7b8-c82d84be948b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.668243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf9f3d-1e3e-4908-b867-7f318519434b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.668366 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7f5b9-b94b-4569-a7b8-c82d84be948b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.668951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf9f3d-1e3e-4908-b867-7f318519434b admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.669318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf9f3d-1e3e-4908-b867-7f318519434b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.670143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf9f3d-1e3e-4908-b867-7f318519434b admin admin] PATH_INFO: `/v3/projects/a214702b5d894605a09e9321e280a5b1/users/11c3485cad2f45fda12277f74801c841/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.683914 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 809/1598] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:09.683914 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7f5b9-b94b-4569-a7b8-c82d84be948b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.687174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf9f3d-1e3e-4908-b867-7f318519434b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a214702b5d894605a09e9321e280a5b1, user_id=11c3485cad2f45fda12277f74801c841, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.689632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf9f3d-1e3e-4908-b867-7f318519434b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.694952 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7f5b9-b94b-4569-a7b8-c82d84be948b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.702121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430287c-7c66-47d1-bc72-f7f141c5f9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.702647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430287c-7c66-47d1-bc72-f7f141c5f9b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.702933 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430287c-7c66-47d1-bc72-f7f141c5f9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.703209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430287c-7c66-47d1-bc72-f7f141c5f9b4 admin admin] PATH_INFO: `/v3/projects/771859448b884c5381529ba3a6137511/users/9bfe75272fa747eb9cdede07291fae9e/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.718550 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 790/1599] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:09 2022] PUT /identity/v3/projects/a214702b5d894605a09e9321e280a5b1/users/11c3485cad2f45fda12277f74801c841/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:09.718550 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf9f3d-1e3e-4908-b867-7f318519434b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.736692 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430287c-7c66-47d1-bc72-f7f141c5f9b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=771859448b884c5381529ba3a6137511, user_id=9bfe75272fa747eb9cdede07291fae9e, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.739616 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430287c-7c66-47d1-bc72-f7f141c5f9b4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.760971 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 810/1600] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:09 2022] PUT /identity/v3/projects/771859448b884c5381529ba3a6137511/users/9bfe75272fa747eb9cdede07291fae9e/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:09.762004 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430287c-7c66-47d1-bc72-f7f141c5f9b4 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.773141 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:09.807552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf9f3d-1e3e-4908-b867-7f318519434b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.843479 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430287c-7c66-47d1-bc72-f7f141c5f9b4 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:09.848504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b40f00-5580-424b-a379-95f61c04a1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.849375 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b40f00-5580-424b-a379-95f61c04a1be admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.849660 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b40f00-5580-424b-a379-95f61c04a1be admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.850061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b40f00-5580-424b-a379-95f61c04a1be admin admin] PATH_INFO: `/v3/projects/e8a54c4498624ea4baf34f732a1730d1/users/78c2026a886744e1bcaab9420d14dc88/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.870010 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b40f00-5580-424b-a379-95f61c04a1be admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8a54c4498624ea4baf34f732a1730d1, user_id=78c2026a886744e1bcaab9420d14dc88, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.872023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239f6c3-2ae1-420a-81e1-db88fc707186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:09.872826 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239f6c3-2ae1-420a-81e1-db88fc707186 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:09.873099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b40f00-5580-424b-a379-95f61c04a1be admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.873221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239f6c3-2ae1-420a-81e1-db88fc707186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:09.873414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239f6c3-2ae1-420a-81e1-db88fc707186 admin admin] PATH_INFO: `/v3/projects/69d2acb456bc4aaeae1ff0927195aba2/users/18d196263d2c49f88b5f81acb2586bc1/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:09.896132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239f6c3-2ae1-420a-81e1-db88fc707186 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d2acb456bc4aaeae1ff0927195aba2, user_id=18d196263d2c49f88b5f81acb2586bc1, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:09.897925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 811/1601] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:09 2022] PUT /identity/v3/projects/e8a54c4498624ea4baf34f732a1730d1/users/78c2026a886744e1bcaab9420d14dc88/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:09.898970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b40f00-5580-424b-a379-95f61c04a1be admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.906013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239f6c3-2ae1-420a-81e1-db88fc707186 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:09.916573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:09.929829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 791/1602] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:09 2022] PUT /identity/v3/projects/69d2acb456bc4aaeae1ff0927195aba2/users/18d196263d2c49f88b5f81acb2586bc1/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:09.931290 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239f6c3-2ae1-420a-81e1-db88fc707186 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:09.991176 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b40f00-5580-424b-a379-95f61c04a1be admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:10.025115 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239f6c3-2ae1-420a-81e1-db88fc707186 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:10.031190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7deb85-ed12-4c6e-944a-772f21a552e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:10.032038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7deb85-ed12-4c6e-944a-772f21a552e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.032747 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7deb85-ed12-4c6e-944a-772f21a552e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.032747 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7deb85-ed12-4c6e-944a-772f21a552e8 admin admin] PATH_INFO: `/v3/projects/33120f340d8f452597ace0cdcba72b8c/users/6c9e75ec32704fd9897e489d633432bf/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.049650 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7deb85-ed12-4c6e-944a-772f21a552e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33120f340d8f452597ace0cdcba72b8c, user_id=6c9e75ec32704fd9897e489d633432bf, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:10.052351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7deb85-ed12-4c6e-944a-772f21a552e8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:10.060353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79cc5c6-1d43-4d7d-a9df-7157b8370126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:10.061025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cc5c6-1d43-4d7d-a9df-7157b8370126 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.061614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cc5c6-1d43-4d7d-a9df-7157b8370126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.061896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cc5c6-1d43-4d7d-a9df-7157b8370126 admin admin] PATH_INFO: `/v3/projects/a4c444b37c884b8aad96c00dda9615b2/users/e3bab8e3620642d1a3e8b9d38d4af0ed/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.068097 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 792/1603] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:09 2022] PUT /identity/v3/projects/33120f340d8f452597ace0cdcba72b8c/users/6c9e75ec32704fd9897e489d633432bf/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:10.068659 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7deb85-ed12-4c6e-944a-772f21a552e8 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:10.078960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79cc5c6-1d43-4d7d-a9df-7157b8370126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4c444b37c884b8aad96c00dda9615b2, user_id=e3bab8e3620642d1a3e8b9d38d4af0ed, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:10.082576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79cc5c6-1d43-4d7d-a9df-7157b8370126 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:10.098995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 812/1604] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:09 2022] PUT /identity/v3/projects/a4c444b37c884b8aad96c00dda9615b2/users/e3bab8e3620642d1a3e8b9d38d4af0ed/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:10.109883 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f9eda-8143-4ac6-92b6-e3ce7a015499 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.109883 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f9eda-8143-4ac6-92b6-e3ce7a015499 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.109883 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f9eda-8143-4ac6-92b6-e3ce7a015499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.126751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:10.133384 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-705f9eda-8143-4ac6-92b6-e3ce7a015499 None None] MFA Rules not processed for user `11c3485cad2f45fda12277f74801c841`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:10.134859 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-705f9eda-8143-4ac6-92b6-e3ce7a015499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:10.144031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7deb85-ed12-4c6e-944a-772f21a552e8 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:10.205817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 813/1605] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:10.206913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfa3ce-1ef2-4ccb-83b3-9adadf878443 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.207204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfa3ce-1ef2-4ccb-83b3-9adadf878443 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.207481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfa3ce-1ef2-4ccb-83b3-9adadf878443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.222814 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5a722-d3cf-4689-b22c-127b4b7e3c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:10.224115 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5a722-d3cf-4689-b22c-127b4b7e3c75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.224669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5a722-d3cf-4689-b22c-127b4b7e3c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.225334 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5a722-d3cf-4689-b22c-127b4b7e3c75 admin admin] PATH_INFO: `/v3/projects/828d3459cdd641d98772383edfe3268f/users/3066c1c9c79942c289b733d886d9fd0f/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.238046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d7bfa3ce-1ef2-4ccb-83b3-9adadf878443 None None] MFA Rules not processed for user `9bfe75272fa747eb9cdede07291fae9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:10.239294 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d7bfa3ce-1ef2-4ccb-83b3-9adadf878443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:10.247326 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5a722-d3cf-4689-b22c-127b4b7e3c75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=828d3459cdd641d98772383edfe3268f, user_id=3066c1c9c79942c289b733d886d9fd0f, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:10.261108 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:10.267190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5a722-d3cf-4689-b22c-127b4b7e3c75 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:10.288611 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 793/1606] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:10 2022] PUT /identity/v3/projects/828d3459cdd641d98772383edfe3268f/users/3066c1c9c79942c289b733d886d9fd0f/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:10.290287 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032f42f-973e-472b-9b74-f0166a03224c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.290747 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032f42f-973e-472b-9b74-f0166a03224c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.291168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032f42f-973e-472b-9b74-f0166a03224c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.334712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6032f42f-973e-472b-9b74-f0166a03224c None None] MFA Rules not processed for user `78c2026a886744e1bcaab9420d14dc88`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:10.336397 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6032f42f-973e-472b-9b74-f0166a03224c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:10.338810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 814/1607] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:10.340458 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070f765-e8d6-49f6-9ee9-4ea737b69f04 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.340607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070f765-e8d6-49f6-9ee9-4ea737b69f04 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.340833 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070f765-e8d6-49f6-9ee9-4ea737b69f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.362973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-2070f765-e8d6-49f6-9ee9-4ea737b69f04 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:10.364395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2070f765-e8d6-49f6-9ee9-4ea737b69f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:10.445154 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 794/1608] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:10.445154 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b04ec-791d-46d1-92f8-03d5941f0059 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.445154 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b04ec-791d-46d1-92f8-03d5941f0059 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.445154 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b04ec-791d-46d1-92f8-03d5941f0059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.462159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 815/1609] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:10.468092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a882ab-9273-4c5d-b510-2875c3daf322 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.469603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a882ab-9273-4c5d-b510-2875c3daf322 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.470111 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a882ab-9273-4c5d-b510-2875c3daf322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.486505 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c72b04ec-791d-46d1-92f8-03d5941f0059 None None] MFA Rules not processed for user `6c9e75ec32704fd9897e489d633432bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:10.487550 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c72b04ec-791d-46d1-92f8-03d5941f0059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:10.519111 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-30a882ab-9273-4c5d-b510-2875c3daf322 None None] MFA Rules not processed for user `e3bab8e3620642d1a3e8b9d38d4af0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:10.520364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-30a882ab-9273-4c5d-b510-2875c3daf322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:10.560339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:10.573542 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 795/1610] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:10.575382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773315b0-05b3-4633-874b-c7d28a96436e None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.576627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773315b0-05b3-4633-874b-c7d28a96436e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.577083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773315b0-05b3-4633-874b-c7d28a96436e None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.586295 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 796/1611] 10.222.0.33 () {54 vars in 961 bytes} [Thu Oct 6 12:22:10 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:22:10.589007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1033eb5-4108-4f23-9ee2-cd58fbaf081c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.589300 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1033eb5-4108-4f23-9ee2-cd58fbaf081c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.589578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1033eb5-4108-4f23-9ee2-cd58fbaf081c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.619906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e1033eb5-4108-4f23-9ee2-cd58fbaf081c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:10.623778 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e1033eb5-4108-4f23-9ee2-cd58fbaf081c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:10.634077 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 816/1612] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:10.638420 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 797/1613] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:10.638602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a882ab-9273-4c5d-b510-2875c3daf322 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:10.640122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1033eb5-4108-4f23-9ee2-cd58fbaf081c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:10.653826 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a882ab-9273-4c5d-b510-2875c3daf322 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:10.660306 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83fe94-42ee-4621-b359-ac44a522c611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:10.660478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83fe94-42ee-4621-b359-ac44a522c611 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.660759 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83fe94-42ee-4621-b359-ac44a522c611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.661140 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83fe94-42ee-4621-b359-ac44a522c611 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.662203 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac83fe94-42ee-4621-b359-ac44a522c611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:10.664686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac83fe94-42ee-4621-b359-ac44a522c611 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:10.706769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 817/1614] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:10 2022] POST /identity/v3/projects => generated 405 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:10.707744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83fe94-42ee-4621-b359-ac44a522c611 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:10.714774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1033eb5-4108-4f23-9ee2-cd58fbaf081c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:10.752260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:10.779378 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e6d34-8061-4960-a62c-50541a8dc861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:10.780400 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e6d34-8061-4960-a62c-50541a8dc861 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.780827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e6d34-8061-4960-a62c-50541a8dc861 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.781267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e6d34-8061-4960-a62c-50541a8dc861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.790905 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83fe94-42ee-4621-b359-ac44a522c611 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:10.795253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa29a67-3dd0-43f6-917c-13a93dae2e78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:10.796083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa29a67-3dd0-43f6-917c-13a93dae2e78 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.797378 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa29a67-3dd0-43f6-917c-13a93dae2e78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.797667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa29a67-3dd0-43f6-917c-13a93dae2e78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.859106 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e6d34-8061-4960-a62c-50541a8dc861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:10.861472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e6d34-8061-4960-a62c-50541a8dc861 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:10.873407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa29a67-3dd0-43f6-917c-13a93dae2e78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:10.874984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 798/1615] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:10.875117 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa29a67-3dd0-43f6-917c-13a93dae2e78 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:10.876285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d350db3-b931-4066-bc88-355e2552dff8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.876643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d350db3-b931-4066-bc88-355e2552dff8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.876989 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d350db3-b931-4066-bc88-355e2552dff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.885618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 818/1616] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:10.887010 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98708823-24c2-4ec5-a6b8-ab6615730b48 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.887288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98708823-24c2-4ec5-a6b8-ab6615730b48 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.887584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98708823-24c2-4ec5-a6b8-ab6615730b48 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.889025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 819/1617] 10.222.0.33 () {54 vars in 961 bytes} [Thu Oct 6 12:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:22:10.894407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98708823-24c2-4ec5-a6b8-ab6615730b48 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:10.908011 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98708823-24c2-4ec5-a6b8-ab6615730b48 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:10.914986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba3123-8de3-48c7-8708-bc97e2e0142c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:10.915748 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba3123-8de3-48c7-8708-bc97e2e0142c admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:10.915905 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba3123-8de3-48c7-8708-bc97e2e0142c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:10.916304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba3123-8de3-48c7-8708-bc97e2e0142c admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:10.917038 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba3123-8de3-48c7-8708-bc97e2e0142c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:10.918954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba3123-8de3-48c7-8708-bc97e2e0142c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:10.971089 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2d350db3-b931-4066-bc88-355e2552dff8 None None] MFA Rules not processed for user `5a46e63d34564d34b6b5752706375798`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:10.972436 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2d350db3-b931-4066-bc88-355e2552dff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:10.974064 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 820/1618] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:10 2022] POST /identity/v3/projects => generated 363 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:10.976910 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba3123-8de3-48c7-8708-bc97e2e0142c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:11.063373 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:11.108330 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 799/1619] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Oct 6 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:11.109595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d350db3-b931-4066-bc88-355e2552dff8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:11.132520 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba3123-8de3-48c7-8708-bc97e2e0142c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:11.148593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3215f-377b-4a7e-9649-352e97e7d30c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:11.149231 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3215f-377b-4a7e-9649-352e97e7d30c service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:11.150258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3215f-377b-4a7e-9649-352e97e7d30c service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:11.150570 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3215f-377b-4a7e-9649-352e97e7d30c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:11.203532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d350db3-b931-4066-bc88-355e2552dff8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:11.209870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba87f2-4e8e-48f4-9026-e16c3811c84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:11.210814 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba87f2-4e8e-48f4-9026-e16c3811c84b admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:11.215920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba87f2-4e8e-48f4-9026-e16c3811c84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:11.216948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba87f2-4e8e-48f4-9026-e16c3811c84b admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:11.217868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba87f2-4e8e-48f4-9026-e16c3811c84b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:11.220286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba87f2-4e8e-48f4-9026-e16c3811c84b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:11.270325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 800/1620] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:11 2022] POST /identity/v3/users => generated 309 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:11.272374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad510ec8-c0f1-4b4c-9b78-f65b19200459 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:11.273397 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad510ec8-c0f1-4b4c-9b78-f65b19200459 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:11.273735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad510ec8-c0f1-4b4c-9b78-f65b19200459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:11.307334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3215f-377b-4a7e-9649-352e97e7d30c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:11.308867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3215f-377b-4a7e-9649-352e97e7d30c service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:11.318034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ad510ec8-c0f1-4b4c-9b78-f65b19200459 None None] MFA Rules not processed for user `5a46e63d34564d34b6b5752706375798`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:11.323462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ad510ec8-c0f1-4b4c-9b78-f65b19200459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:11.332571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 821/1621] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:11.334480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3215f-377b-4a7e-9649-352e97e7d30c service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:11.349287 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 801/1622] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Oct 6 12:22:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:11.350084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad510ec8-c0f1-4b4c-9b78-f65b19200459 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:11.352187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:11.360740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3215f-377b-4a7e-9649-352e97e7d30c service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:11.369567 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1e8d5-fda8-4dec-82a1-9e79199f9144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:11.370734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1e8d5-fda8-4dec-82a1-9e79199f9144 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:11.371391 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1e8d5-fda8-4dec-82a1-9e79199f9144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:11.372042 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1e8d5-fda8-4dec-82a1-9e79199f9144 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:11.373161 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad510ec8-c0f1-4b4c-9b78-f65b19200459 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:11.373513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff1e8d5-fda8-4dec-82a1-9e79199f9144 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:11.376731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff1e8d5-fda8-4dec-82a1-9e79199f9144 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:11.389654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080928fd-4196-42dc-bd45-933a133c917a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:11.390535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080928fd-4196-42dc-bd45-933a133c917a service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:11.390888 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080928fd-4196-42dc-bd45-933a133c917a service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:11.391232 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080928fd-4196-42dc-bd45-933a133c917a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:11.411769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 822/1623] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:11 2022] POST /identity/v3/users => generated 288 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:11.413062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1e8d5-fda8-4dec-82a1-9e79199f9144 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:11.427384 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1e8d5-fda8-4dec-82a1-9e79199f9144 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:11.433206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce40b0-49cf-4ae9-a27c-41b3222ee9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:11.434176 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce40b0-49cf-4ae9-a27c-41b3222ee9ac admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:11.434758 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce40b0-49cf-4ae9-a27c-41b3222ee9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:11.435184 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce40b0-49cf-4ae9-a27c-41b3222ee9ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:11.436138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ce40b0-49cf-4ae9-a27c-41b3222ee9ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:11.438297 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ce40b0-49cf-4ae9-a27c-41b3222ee9ac admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:11.456953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 823/1624] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:11.456953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce40b0-49cf-4ae9-a27c-41b3222ee9ac admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:11.471565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce40b0-49cf-4ae9-a27c-41b3222ee9ac admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:11.480230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b0acb-63e5-4eef-9097-0344d5bac466 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:11.481219 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b0acb-63e5-4eef-9097-0344d5bac466 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:11.481911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b0acb-63e5-4eef-9097-0344d5bac466 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:11.482333 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b0acb-63e5-4eef-9097-0344d5bac466 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:11.542548 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080928fd-4196-42dc-bd45-933a133c917a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:11.547614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080928fd-4196-42dc-bd45-933a133c917a service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:11.569214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 802/1625] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:11.570596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080928fd-4196-42dc-bd45-933a133c917a service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:11.587993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080928fd-4196-42dc-bd45-933a133c917a service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:11.596829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c512d4b-9cf0-4e1b-84eb-4041d15d86ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:11.598622 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c512d4b-9cf0-4e1b-84eb-4041d15d86ce admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:11.599012 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c512d4b-9cf0-4e1b-84eb-4041d15d86ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:11.599387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c512d4b-9cf0-4e1b-84eb-4041d15d86ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:11.600525 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c512d4b-9cf0-4e1b-84eb-4041d15d86ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:11.602789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c512d4b-9cf0-4e1b-84eb-4041d15d86ce admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:11.621705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 803/1626] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:11.627598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c512d4b-9cf0-4e1b-84eb-4041d15d86ce admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:11.643208 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b0acb-63e5-4eef-9097-0344d5bac466 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:11.649416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b0acb-63e5-4eef-9097-0344d5bac466 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:11.654500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c512d4b-9cf0-4e1b-84eb-4041d15d86ce admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:11.661119 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975d7e4-9f5c-4e7d-ba00-f1188a85ad92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:11.662068 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975d7e4-9f5c-4e7d-ba00-f1188a85ad92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:11.662504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975d7e4-9f5c-4e7d-ba00-f1188a85ad92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:11.662882 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975d7e4-9f5c-4e7d-ba00-f1188a85ad92 admin admin] PATH_INFO: `/v3/projects/cc64e6bd0bfd48be9abf02404e59a734/users/cd451b182ea84b93be6198e7a1640cae/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:11.686161 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 824/1627] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:11.687273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975d7e4-9f5c-4e7d-ba00-f1188a85ad92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc64e6bd0bfd48be9abf02404e59a734, user_id=cd451b182ea84b93be6198e7a1640cae, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:11.688497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b0acb-63e5-4eef-9097-0344d5bac466 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:11.691478 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975d7e4-9f5c-4e7d-ba00-f1188a85ad92 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:11.720927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b0acb-63e5-4eef-9097-0344d5bac466 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:11.728277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 804/1628] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:11 2022] PUT /identity/v3/projects/cc64e6bd0bfd48be9abf02404e59a734/users/cd451b182ea84b93be6198e7a1640cae/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:11.731088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975d7e4-9f5c-4e7d-ba00-f1188a85ad92 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:11.824074 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:11.899731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bdf34-6707-45e4-b56f-e6917c0063c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:11.900410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bdf34-6707-45e4-b56f-e6917c0063c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:11.900693 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bdf34-6707-45e4-b56f-e6917c0063c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:11.901046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bdf34-6707-45e4-b56f-e6917c0063c3 admin admin] PATH_INFO: `/v3/projects/e3d5d86f3ba34f3ea4fec1821103e1fc/users/810939fd49084e9ba9b12e933a386178/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:11.938311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bdf34-6707-45e4-b56f-e6917c0063c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d5d86f3ba34f3ea4fec1821103e1fc, user_id=810939fd49084e9ba9b12e933a386178, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:11.943749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bdf34-6707-45e4-b56f-e6917c0063c3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:11.947282 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975d7e4-9f5c-4e7d-ba00-f1188a85ad92 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:11.962444 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e5e3f-a243-4cbd-976e-bb4523d4c0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:11.963619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e5e3f-a243-4cbd-976e-bb4523d4c0cd admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:11.964041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e5e3f-a243-4cbd-976e-bb4523d4c0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:11.964760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e5e3f-a243-4cbd-976e-bb4523d4c0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:11.966223 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e5e3f-a243-4cbd-976e-bb4523d4c0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:11.975485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 825/1629] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:11 2022] PUT /identity/v3/projects/e3d5d86f3ba34f3ea4fec1821103e1fc/users/810939fd49084e9ba9b12e933a386178/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:11.979187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bdf34-6707-45e4-b56f-e6917c0063c3 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:11.979887 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e5e3f-a243-4cbd-976e-bb4523d4c0cd admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:12.085592 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 805/1630] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:12.085592 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e5e3f-a243-4cbd-976e-bb4523d4c0cd admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:12.097291 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e5e3f-a243-4cbd-976e-bb4523d4c0cd admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:12.208582 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74efb2b-b7e0-4033-b9b0-ed95890e0095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:12.209367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74efb2b-b7e0-4033-b9b0-ed95890e0095 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:12.209772 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74efb2b-b7e0-4033-b9b0-ed95890e0095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:12.211045 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74efb2b-b7e0-4033-b9b0-ed95890e0095 admin admin] PATH_INFO: `/v3/projects/cc64e6bd0bfd48be9abf02404e59a734/users/cd451b182ea84b93be6198e7a1640cae/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:12.224646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74efb2b-b7e0-4033-b9b0-ed95890e0095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc64e6bd0bfd48be9abf02404e59a734, user_id=cd451b182ea84b93be6198e7a1640cae, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:12.225216 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:12.228339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74efb2b-b7e0-4033-b9b0-ed95890e0095 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:12.234590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bdf34-6707-45e4-b56f-e6917c0063c3 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:12.258427 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231304f-640f-4f22-8b4c-b5ac262cab01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:12.259761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231304f-640f-4f22-8b4c-b5ac262cab01 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:12.260066 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231304f-640f-4f22-8b4c-b5ac262cab01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:12.260342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231304f-640f-4f22-8b4c-b5ac262cab01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:12.260996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231304f-640f-4f22-8b4c-b5ac262cab01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:12.269076 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231304f-640f-4f22-8b4c-b5ac262cab01 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:12.283886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 806/1631] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:12 2022] PUT /identity/v3/projects/cc64e6bd0bfd48be9abf02404e59a734/users/cd451b182ea84b93be6198e7a1640cae/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:12.285931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19437a5-b3af-4ac6-9da1-6a24f88646cc None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:12.286328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19437a5-b3af-4ac6-9da1-6a24f88646cc None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:12.286765 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19437a5-b3af-4ac6-9da1-6a24f88646cc None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:12.288871 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 807/1632] 10.222.0.33 () {54 vars in 912 bytes} [Thu Oct 6 12:22:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:22:12.290183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5e451-a901-4b66-8ee4-4da925526df1 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:12.290544 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5e451-a901-4b66-8ee4-4da925526df1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:12.292111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5e451-a901-4b66-8ee4-4da925526df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:12.304744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 826/1633] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:12.314465 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3361fe-4ea3-4707-a568-a4c418e5d0c1 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:12.314772 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3361fe-4ea3-4707-a568-a4c418e5d0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:12.315037 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3361fe-4ea3-4707-a568-a4c418e5d0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:12.352001 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-aad5e451-a901-4b66-8ee4-4da925526df1 None None] MFA Rules not processed for user `cd451b182ea84b93be6198e7a1640cae`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:12.353514 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-aad5e451-a901-4b66-8ee4-4da925526df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:12.387922 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-9d3361fe-4ea3-4707-a568-a4c418e5d0c1 None None] MFA Rules not processed for user `5a46e63d34564d34b6b5752706375798`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:12.390482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9d3361fe-4ea3-4707-a568-a4c418e5d0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:12.475746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 808/1634] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:12 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:12.481102 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5e451-a901-4b66-8ee4-4da925526df1 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:12.490389 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 827/1635] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Oct 6 12:22:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:12.491351 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5cf9f-a8d1-4799-9721-a83e5c5499e2 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:12.493730 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5cf9f-a8d1-4799-9721-a83e5c5499e2 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:12.494012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5cf9f-a8d1-4799-9721-a83e5c5499e2 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:12.495448 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 828/1636] 10.222.0.33 () {54 vars in 912 bytes} [Thu Oct 6 12:22:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:22:12.497244 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5cf9f-a8d1-4799-9721-a83e5c5499e2 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:12.561337 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:12.596013 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5cf9f-a8d1-4799-9721-a83e5c5499e2 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:12.597658 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5e451-a901-4b66-8ee4-4da925526df1 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:12.601133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a5031-042a-4067-beb1-e411cd76d366 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:12.601881 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a5031-042a-4067-beb1-e411cd76d366 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:12.602298 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a5031-042a-4067-beb1-e411cd76d366 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:12.602580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a5031-042a-4067-beb1-e411cd76d366 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:12.603636 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd8cea-0439-4dfb-a43d-7953acc79b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:12.604580 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd8cea-0439-4dfb-a43d-7953acc79b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:12.605066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd8cea-0439-4dfb-a43d-7953acc79b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:12.605413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd8cea-0439-4dfb-a43d-7953acc79b13 admin admin] PATH_INFO: `/v3/projects/e3d5d86f3ba34f3ea4fec1821103e1fc/users/810939fd49084e9ba9b12e933a386178/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:12.608406 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd8cea-0439-4dfb-a43d-7953acc79b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d5d86f3ba34f3ea4fec1821103e1fc, user_id=810939fd49084e9ba9b12e933a386178, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:12.611402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd8cea-0439-4dfb-a43d-7953acc79b13 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:12.614157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a5031-042a-4067-beb1-e411cd76d366 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:12.615900 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a5031-042a-4067-beb1-e411cd76d366 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:12.634892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 809/1637] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:12 2022] PUT /identity/v3/projects/e3d5d86f3ba34f3ea4fec1821103e1fc/users/810939fd49084e9ba9b12e933a386178/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:12.636913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2c716-5758-4a75-8f36-d4ae5ab40ed2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:12.637375 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2c716-5758-4a75-8f36-d4ae5ab40ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:12.637727 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2c716-5758-4a75-8f36-d4ae5ab40ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:12.643836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 829/1638] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:12 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:12.645698 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a5031-042a-4067-beb1-e411cd76d366 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:12.664902 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-66f2c716-5758-4a75-8f36-d4ae5ab40ed2 None None] MFA Rules not processed for user `5a46e63d34564d34b6b5752706375798`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:12.667019 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-66f2c716-5758-4a75-8f36-d4ae5ab40ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:12.679223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a5031-042a-4067-beb1-e411cd76d366 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:12.804239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b79e7d-206e-4194-b64e-c1127ffbf5f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:12.806137 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b79e7d-206e-4194-b64e-c1127ffbf5f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:12.807136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b79e7d-206e-4194-b64e-c1127ffbf5f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:12.807942 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b79e7d-206e-4194-b64e-c1127ffbf5f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:12.846648 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 810/1639] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Oct 6 12:22:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:12.849353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d25a8-02db-404e-b5c8-a16459e76acd None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:12.850052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d25a8-02db-404e-b5c8-a16459e76acd None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:12.852452 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d25a8-02db-404e-b5c8-a16459e76acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:12.921891 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:12.922927 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-247d25a8-02db-404e-b5c8-a16459e76acd None None] MFA Rules not processed for user `810939fd49084e9ba9b12e933a386178`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:12.925603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-247d25a8-02db-404e-b5c8-a16459e76acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:12.967418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b79e7d-206e-4194-b64e-c1127ffbf5f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:12.970695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b79e7d-206e-4194-b64e-c1127ffbf5f4 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:12.989126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 830/1640] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:12.990247 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b79e7d-206e-4194-b64e-c1127ffbf5f4 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:13.051380 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 811/1641] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:13.054856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d25a8-02db-404e-b5c8-a16459e76acd None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:13.079127 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d25a8-02db-404e-b5c8-a16459e76acd None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:13.092187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60865c4-6845-4144-9c8f-69e388a346d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:13.092939 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60865c4-6845-4144-9c8f-69e388a346d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:13.093496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60865c4-6845-4144-9c8f-69e388a346d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:13.094393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60865c4-6845-4144-9c8f-69e388a346d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:13.114816 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b79e7d-206e-4194-b64e-c1127ffbf5f4 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:13.125631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b9506-8c29-429f-bc79-ad9a6b8cd65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:13.126556 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b9506-8c29-429f-bc79-ad9a6b8cd65b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:13.126713 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b9506-8c29-429f-bc79-ad9a6b8cd65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:13.126990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b9506-8c29-429f-bc79-ad9a6b8cd65b admin admin] PATH_INFO: `/v3/users/9bfe75272fa747eb9cdede07291fae9e` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:13.129423 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60865c4-6845-4144-9c8f-69e388a346d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:13.131180 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60865c4-6845-4144-9c8f-69e388a346d4 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:13.132951 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b9506-8c29-429f-bc79-ad9a6b8cd65b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bfe75272fa747eb9cdede07291fae9e)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:13.137860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: WARNING py.warnings [None req-ad2b9506-8c29-429f-bc79-ad9a6b8cd65b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 12:22:13.137860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: warnings.warn(msg) Oct 06 12:22:13.137860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: Oct 06 12:22:13.142918 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b9506-8c29-429f-bc79-ad9a6b8cd65b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:13.153024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 812/1642] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:13.279620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ad2b9506-8c29-429f-bc79-ad9a6b8cd65b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bfe75272fa747eb9cdede07291fae9e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:13.307246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60865c4-6845-4144-9c8f-69e388a346d4 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:13.318213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60865c4-6845-4144-9c8f-69e388a346d4 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:13.331092 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f21741-0b56-4e8b-af18-5484dab025b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:13.332000 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f21741-0b56-4e8b-af18-5484dab025b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:13.332281 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f21741-0b56-4e8b-af18-5484dab025b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:13.332552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f21741-0b56-4e8b-af18-5484dab025b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:13.333056 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ad2b9506-8c29-429f-bc79-ad9a6b8cd65b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bfe75272fa747eb9cdede07291fae9e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:13.334345 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ad2b9506-8c29-429f-bc79-ad9a6b8cd65b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bfe75272fa747eb9cdede07291fae9e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:13.335132 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ad2b9506-8c29-429f-bc79-ad9a6b8cd65b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bfe75272fa747eb9cdede07291fae9e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:13.336929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c8f21741-0b56-4e8b-af18-5484dab025b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:13.352431 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ad2b9506-8c29-429f-bc79-ad9a6b8cd65b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bfe75272fa747eb9cdede07291fae9e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:13.367781 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f21741-0b56-4e8b-af18-5484dab025b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:13.370566 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f21741-0b56-4e8b-af18-5484dab025b9 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:13.422857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 813/1643] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:22:13.439633 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 831/1644] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:12 2022] DELETE /identity/v3/users/9bfe75272fa747eb9cdede07291fae9e => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:13.444254 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd056a-4371-4acf-a413-5f435e5e4f88 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:13.444829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd056a-4371-4acf-a413-5f435e5e4f88 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:13.445246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd056a-4371-4acf-a413-5f435e5e4f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:13.487540 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-25dd056a-4371-4acf-a413-5f435e5e4f88 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:13.489943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-25dd056a-4371-4acf-a413-5f435e5e4f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:13.608596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:13.760710 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 814/1645] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:13.795097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b9506-8c29-429f-bc79-ad9a6b8cd65b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:13.796585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ad2b9506-8c29-429f-bc79-ad9a6b8cd65b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:13.848602 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd056a-4371-4acf-a413-5f435e5e4f88 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:13.852660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-25dd056a-4371-4acf-a413-5f435e5e4f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:13.973393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:13.979399 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b9506-8c29-429f-bc79-ad9a6b8cd65b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:13.985364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421c71a-f544-47b5-8f4e-5c640d1109ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:13.986241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421c71a-f544-47b5-8f4e-5c640d1109ef service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:13.987178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421c71a-f544-47b5-8f4e-5c640d1109ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:13.988397 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421c71a-f544-47b5-8f4e-5c640d1109ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:14.004405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421c71a-f544-47b5-8f4e-5c640d1109ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:14.007401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421c71a-f544-47b5-8f4e-5c640d1109ef service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:14.010605 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd056a-4371-4acf-a413-5f435e5e4f88 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:14.024926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32819c8-c16a-4ab6-9d03-8188a63cde4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:14.025569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32819c8-c16a-4ab6-9d03-8188a63cde4c service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:14.025854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32819c8-c16a-4ab6-9d03-8188a63cde4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:14.026130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32819c8-c16a-4ab6-9d03-8188a63cde4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:14.027798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 832/1646] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:14.028840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421c71a-f544-47b5-8f4e-5c640d1109ef service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:14.032294 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8421c71a-f544-47b5-8f4e-5c640d1109ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:14.037680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f32819c8-c16a-4ab6-9d03-8188a63cde4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:14.062086 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421c71a-f544-47b5-8f4e-5c640d1109ef service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:14.082802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee003871-859f-41e9-92cc-1e8d6f2fd599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:14.083642 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee003871-859f-41e9-92cc-1e8d6f2fd599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:14.084000 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee003871-859f-41e9-92cc-1e8d6f2fd599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:14.084324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee003871-859f-41e9-92cc-1e8d6f2fd599 admin admin] PATH_INFO: `/v3/users/6c9e75ec32704fd9897e489d633432bf` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:14.085677 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee003871-859f-41e9-92cc-1e8d6f2fd599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c9e75ec32704fd9897e489d633432bf)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:14.090868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee003871-859f-41e9-92cc-1e8d6f2fd599 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:14.191711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ee003871-859f-41e9-92cc-1e8d6f2fd599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c9e75ec32704fd9897e489d633432bf'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:14.204486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32819c8-c16a-4ab6-9d03-8188a63cde4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:14.206627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32819c8-c16a-4ab6-9d03-8188a63cde4c service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:14.209953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ee003871-859f-41e9-92cc-1e8d6f2fd599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c9e75ec32704fd9897e489d633432bf'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:14.212531 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ee003871-859f-41e9-92cc-1e8d6f2fd599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c9e75ec32704fd9897e489d633432bf'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:14.213774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ee003871-859f-41e9-92cc-1e8d6f2fd599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c9e75ec32704fd9897e489d633432bf'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:14.229863 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 815/1647] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:14.230287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ee003871-859f-41e9-92cc-1e8d6f2fd599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c9e75ec32704fd9897e489d633432bf'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:14.269014 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 833/1648] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:14 2022] DELETE /identity/v3/users/6c9e75ec32704fd9897e489d633432bf => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:14.273033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbae30-ca03-4be7-b1ef-3761820992db None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:14.273327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbae30-ca03-4be7-b1ef-3761820992db None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:14.273607 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbae30-ca03-4be7-b1ef-3761820992db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:14.320701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-bdcbae30-ca03-4be7-b1ef-3761820992db None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:14.322047 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bdcbae30-ca03-4be7-b1ef-3761820992db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:14.465806 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 816/1649] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:14.495172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee003871-859f-41e9-92cc-1e8d6f2fd599 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:14.497658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ee003871-859f-41e9-92cc-1e8d6f2fd599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:14.661478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee003871-859f-41e9-92cc-1e8d6f2fd599 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:14.673349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045ca20-e9d1-43f9-8425-f07ee4e4a484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:14.676337 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045ca20-e9d1-43f9-8425-f07ee4e4a484 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:14.676664 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045ca20-e9d1-43f9-8425-f07ee4e4a484 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:14.679245 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045ca20-e9d1-43f9-8425-f07ee4e4a484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:14.710513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045ca20-e9d1-43f9-8425-f07ee4e4a484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:14.714571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045ca20-e9d1-43f9-8425-f07ee4e4a484 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:14.748044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 834/1650] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:15.414225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbae30-ca03-4be7-b1ef-3761820992db None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:15.416901 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bdcbae30-ca03-4be7-b1ef-3761820992db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:15.441234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045ca20-e9d1-43f9-8425-f07ee4e4a484 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:15.443134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3045ca20-e9d1-43f9-8425-f07ee4e4a484 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:15.456578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbae30-ca03-4be7-b1ef-3761820992db None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:15.472921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea04cf4-c39a-459c-9782-b857c1ac354d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:15.474395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045ca20-e9d1-43f9-8425-f07ee4e4a484 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:15.478185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea04cf4-c39a-459c-9782-b857c1ac354d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:15.478958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea04cf4-c39a-459c-9782-b857c1ac354d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:15.480020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea04cf4-c39a-459c-9782-b857c1ac354d admin admin] PATH_INFO: `/v3/projects/771859448b884c5381529ba3a6137511` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:15.486264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea04cf4-c39a-459c-9782-b857c1ac354d admin admin] RBAC: Authorizing `identity:delete_project(project_id=771859448b884c5381529ba3a6137511)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:15.488396 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: WARNING py.warnings [None req-cea04cf4-c39a-459c-9782-b857c1ac354d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 12:22:15.488396 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: warnings.warn(msg) Oct 06 12:22:15.488396 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: Oct 06 12:22:15.493395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea04cf4-c39a-459c-9782-b857c1ac354d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:15.498860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd929d-4d63-4221-8666-916f799b2e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:15.501858 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd929d-4d63-4221-8666-916f799b2e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:15.502405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd929d-4d63-4221-8666-916f799b2e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:15.503279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd929d-4d63-4221-8666-916f799b2e94 admin admin] PATH_INFO: `/v3/projects/33120f340d8f452597ace0cdcba72b8c` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:15.508755 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd929d-4d63-4221-8666-916f799b2e94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33120f340d8f452597ace0cdcba72b8c)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:15.515467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd929d-4d63-4221-8666-916f799b2e94 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:15.557688 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:15.681426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3cfd929d-4d63-4221-8666-916f799b2e94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33120f340d8f452597ace0cdcba72b8c', 'request_id': 'req-3cfd929d-4d63-4221-8666-916f799b2e94'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:15.686084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cea04cf4-c39a-459c-9782-b857c1ac354d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '771859448b884c5381529ba3a6137511', 'request_id': 'req-cea04cf4-c39a-459c-9782-b857c1ac354d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:15.695473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3cfd929d-4d63-4221-8666-916f799b2e94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33120f340d8f452597ace0cdcba72b8c', 'request_id': 'req-3cfd929d-4d63-4221-8666-916f799b2e94'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:15.705648 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cea04cf4-c39a-459c-9782-b857c1ac354d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '771859448b884c5381529ba3a6137511', 'request_id': 'req-cea04cf4-c39a-459c-9782-b857c1ac354d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:15.706399 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3cfd929d-4d63-4221-8666-916f799b2e94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33120f340d8f452597ace0cdcba72b8c', 'request_id': 'req-3cfd929d-4d63-4221-8666-916f799b2e94'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:15.710336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:15.730283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cea04cf4-c39a-459c-9782-b857c1ac354d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '771859448b884c5381529ba3a6137511', 'request_id': 'req-cea04cf4-c39a-459c-9782-b857c1ac354d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:15.740665 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3cfd929d-4d63-4221-8666-916f799b2e94 admin admin] The token cache is being invalidate because project 33120f340d8f452597ace0cdcba72b8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:15.740772 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3cfd929d-4d63-4221-8666-916f799b2e94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:15.741431 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3cfd929d-4d63-4221-8666-916f799b2e94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:15.743265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 835/1651] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:15 2022] DELETE /identity/v3/projects/33120f340d8f452597ace0cdcba72b8c => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:15.749269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cea04cf4-c39a-459c-9782-b857c1ac354d admin admin] The token cache is being invalidate because project 771859448b884c5381529ba3a6137511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:15.749513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5ed6d-f425-42a6-8bc5-454b46ee69d3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:15.750103 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5ed6d-f425-42a6-8bc5-454b46ee69d3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:15.750103 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5ed6d-f425-42a6-8bc5-454b46ee69d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:15.750790 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cea04cf4-c39a-459c-9782-b857c1ac354d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:15.752529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cea04cf4-c39a-459c-9782-b857c1ac354d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:15.756537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 817/1652] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:15 2022] DELETE /identity/v3/projects/771859448b884c5381529ba3a6137511 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:15.760320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea04cf4-c39a-459c-9782-b857c1ac354d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:15.762823 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-cea04cf4-c39a-459c-9782-b857c1ac354d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:15.808929 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e8e5ed6d-f425-42a6-8bc5-454b46ee69d3 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:15.810985 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e8e5ed6d-f425-42a6-8bc5-454b46ee69d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:15.962125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea04cf4-c39a-459c-9782-b857c1ac354d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:15.986336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 836/1653] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:16.014878 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c4567-e560-4850-863b-a23433f3dd89 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:16.016144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c4567-e560-4850-863b-a23433f3dd89 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:16.016645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c4567-e560-4850-863b-a23433f3dd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:16.096341 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe8350-db22-4096-810a-fd81e69fe646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:16.097011 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe8350-db22-4096-810a-fd81e69fe646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:16.097313 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe8350-db22-4096-810a-fd81e69fe646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:16.097648 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe8350-db22-4096-810a-fd81e69fe646 admin admin] PATH_INFO: `/v3/users/cd451b182ea84b93be6198e7a1640cae` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:16.104217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-eb9c4567-e560-4850-863b-a23433f3dd89 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:16.106530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-eb9c4567-e560-4850-863b-a23433f3dd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:16.108568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe8350-db22-4096-810a-fd81e69fe646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd451b182ea84b93be6198e7a1640cae)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:16.110567 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe8350-db22-4096-810a-fd81e69fe646 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:16.166977 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 837/1654] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:16.169316 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c4567-e560-4850-863b-a23433f3dd89 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:16.229989 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-51fe8350-db22-4096-810a-fd81e69fe646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd451b182ea84b93be6198e7a1640cae'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:16.262377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:16.270537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-51fe8350-db22-4096-810a-fd81e69fe646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd451b182ea84b93be6198e7a1640cae'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:16.271605 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-51fe8350-db22-4096-810a-fd81e69fe646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd451b182ea84b93be6198e7a1640cae'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:16.272510 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-51fe8350-db22-4096-810a-fd81e69fe646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd451b182ea84b93be6198e7a1640cae'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:16.275481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c4567-e560-4850-863b-a23433f3dd89 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:16.297952 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-51fe8350-db22-4096-810a-fd81e69fe646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd451b182ea84b93be6198e7a1640cae'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:16.307846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd813c-77d9-4a53-b029-caabb41648b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:16.315343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd813c-77d9-4a53-b029-caabb41648b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:16.317490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd813c-77d9-4a53-b029-caabb41648b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:16.317789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd813c-77d9-4a53-b029-caabb41648b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:16.323045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd813c-77d9-4a53-b029-caabb41648b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:16.323465 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd813c-77d9-4a53-b029-caabb41648b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:16.336684 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd813c-77d9-4a53-b029-caabb41648b5 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:16.343734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 818/1655] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:15 2022] DELETE /identity/v3/users/cd451b182ea84b93be6198e7a1640cae => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:16.394606 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 838/1656] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:16.396358 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe8350-db22-4096-810a-fd81e69fe646 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:16.397399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-51fe8350-db22-4096-810a-fd81e69fe646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:16.529822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe8350-db22-4096-810a-fd81e69fe646 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:16.537365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583526a-e06b-47ad-835f-53bf0ef90a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:16.538468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583526a-e06b-47ad-835f-53bf0ef90a39 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:16.539387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583526a-e06b-47ad-835f-53bf0ef90a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:16.540088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583526a-e06b-47ad-835f-53bf0ef90a39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:16.540964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d583526a-e06b-47ad-835f-53bf0ef90a39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:16.543487 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d583526a-e06b-47ad-835f-53bf0ef90a39 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:16.595680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 819/1657] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:16 2022] POST /identity/v3/projects => generated 355 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:16.595789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd813c-77d9-4a53-b029-caabb41648b5 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:16.609342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd813c-77d9-4a53-b029-caabb41648b5 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:16.871664 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c4c63-e89b-4a2d-8a86-48e5da692272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:16.875670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c4c63-e89b-4a2d-8a86-48e5da692272 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:16.875966 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c4c63-e89b-4a2d-8a86-48e5da692272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:16.876957 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c4c63-e89b-4a2d-8a86-48e5da692272 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:16.879534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c4c63-e89b-4a2d-8a86-48e5da692272 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:16.885958 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c4c63-e89b-4a2d-8a86-48e5da692272 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:16.957159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 839/1658] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:16 2022] POST /identity/v3/users => generated 278 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:22:16.962414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583526a-e06b-47ad-835f-53bf0ef90a39 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:16.980459 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c4c63-e89b-4a2d-8a86-48e5da692272 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:16.984023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583526a-e06b-47ad-835f-53bf0ef90a39 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:16.986686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-197c4c63-e89b-4a2d-8a86-48e5da692272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:16.996138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15453d4-3844-4813-ba42-e1c0d6bc5738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:16.998660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15453d4-3844-4813-ba42-e1c0d6bc5738 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:16.998951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15453d4-3844-4813-ba42-e1c0d6bc5738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:17.000902 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15453d4-3844-4813-ba42-e1c0d6bc5738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:17.002940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15453d4-3844-4813-ba42-e1c0d6bc5738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:17.009119 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15453d4-3844-4813-ba42-e1c0d6bc5738 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:17.030028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c4c63-e89b-4a2d-8a86-48e5da692272 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:17.040740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be77e93-8057-40b1-959e-5d9a954d2c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:17.043125 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be77e93-8057-40b1-959e-5d9a954d2c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:17.043449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be77e93-8057-40b1-959e-5d9a954d2c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:17.043902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be77e93-8057-40b1-959e-5d9a954d2c8b admin admin] PATH_INFO: `/v3/users/11c3485cad2f45fda12277f74801c841` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:17.046906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be77e93-8057-40b1-959e-5d9a954d2c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=11c3485cad2f45fda12277f74801c841)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:17.052227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be77e93-8057-40b1-959e-5d9a954d2c8b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:17.079451 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 820/1659] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:17.088293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15453d4-3844-4813-ba42-e1c0d6bc5738 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:17.137596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15453d4-3844-4813-ba42-e1c0d6bc5738 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:17.153399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209dc35a-4494-4bf3-b3b4-a94bb480c507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:17.157925 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dc35a-4494-4bf3-b3b4-a94bb480c507 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:17.158582 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dc35a-4494-4bf3-b3b4-a94bb480c507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:17.161498 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dc35a-4494-4bf3-b3b4-a94bb480c507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:17.163017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209dc35a-4494-4bf3-b3b4-a94bb480c507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:17.171733 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209dc35a-4494-4bf3-b3b4-a94bb480c507 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:17.210141 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8be77e93-8057-40b1-959e-5d9a954d2c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11c3485cad2f45fda12277f74801c841'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:17.229093 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:17.229776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 821/1660] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:17.230978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8be77e93-8057-40b1-959e-5d9a954d2c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11c3485cad2f45fda12277f74801c841'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:17.232504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8be77e93-8057-40b1-959e-5d9a954d2c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11c3485cad2f45fda12277f74801c841'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:17.233320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209dc35a-4494-4bf3-b3b4-a94bb480c507 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:17.235748 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8be77e93-8057-40b1-959e-5d9a954d2c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11c3485cad2f45fda12277f74801c841'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:17.246861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-209dc35a-4494-4bf3-b3b4-a94bb480c507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:17.265721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8be77e93-8057-40b1-959e-5d9a954d2c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11c3485cad2f45fda12277f74801c841'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:17.279479 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209dc35a-4494-4bf3-b3b4-a94bb480c507 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:17.301055 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a252cf-1500-4287-9e86-4a63173c68d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:17.301828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a252cf-1500-4287-9e86-4a63173c68d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:17.302148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a252cf-1500-4287-9e86-4a63173c68d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:17.302431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a252cf-1500-4287-9e86-4a63173c68d2 admin admin] PATH_INFO: `/v3/projects/cb99e62a65bc47f6804ccdc2534c244e/users/a7f055ba26ea4b6db60108b6ce66514e/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:17.322797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 840/1661] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:16 2022] DELETE /identity/v3/users/11c3485cad2f45fda12277f74801c841 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:17.325134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691fc80-f5b5-4283-9075-dcd6f020ac34 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:17.326581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691fc80-f5b5-4283-9075-dcd6f020ac34 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:17.326864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691fc80-f5b5-4283-9075-dcd6f020ac34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:17.348453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a252cf-1500-4287-9e86-4a63173c68d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb99e62a65bc47f6804ccdc2534c244e, user_id=a7f055ba26ea4b6db60108b6ce66514e, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:17.352557 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a252cf-1500-4287-9e86-4a63173c68d2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:17.385040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 822/1662] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:17 2022] PUT /identity/v3/projects/cb99e62a65bc47f6804ccdc2534c244e/users/a7f055ba26ea4b6db60108b6ce66514e/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:17.396740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364860a-2df1-4b3d-adf3-04fc2f4dd7d2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:17.397209 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364860a-2df1-4b3d-adf3-04fc2f4dd7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:17.397803 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364860a-2df1-4b3d-adf3-04fc2f4dd7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:17.403929 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-5691fc80-f5b5-4283-9075-dcd6f020ac34 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:17.406632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5691fc80-f5b5-4283-9075-dcd6f020ac34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:17.452461 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a364860a-2df1-4b3d-adf3-04fc2f4dd7d2 None None] MFA Rules not processed for user `a7f055ba26ea4b6db60108b6ce66514e`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:17.454647 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a364860a-2df1-4b3d-adf3-04fc2f4dd7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:17.560634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 841/1663] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:17.606478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691fc80-f5b5-4283-9075-dcd6f020ac34 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:17.608695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5691fc80-f5b5-4283-9075-dcd6f020ac34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:17.613100 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:17.638268 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 823/1664] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:22:17.642572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364860a-2df1-4b3d-adf3-04fc2f4dd7d2 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:17.648309 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a364860a-2df1-4b3d-adf3-04fc2f4dd7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:17.696206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364860a-2df1-4b3d-adf3-04fc2f4dd7d2 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:17.711808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87895ee-4ce6-484e-870a-abcdab6607bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:17.712841 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87895ee-4ce6-484e-870a-abcdab6607bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:17.713313 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87895ee-4ce6-484e-870a-abcdab6607bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:17.713785 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87895ee-4ce6-484e-870a-abcdab6607bd admin admin] PATH_INFO: `/v3/projects/cc64e6bd0bfd48be9abf02404e59a734` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:17.716883 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87895ee-4ce6-484e-870a-abcdab6607bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc64e6bd0bfd48be9abf02404e59a734)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:17.719041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87895ee-4ce6-484e-870a-abcdab6607bd admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:17.762458 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691fc80-f5b5-4283-9075-dcd6f020ac34 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:17.766945 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d39bbd-20e7-4289-b32c-3041ed4595ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:17.767617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d39bbd-20e7-4289-b32c-3041ed4595ad service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:17.767979 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d39bbd-20e7-4289-b32c-3041ed4595ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:17.768261 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d39bbd-20e7-4289-b32c-3041ed4595ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:17.780589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d39bbd-20e7-4289-b32c-3041ed4595ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:17.782649 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d39bbd-20e7-4289-b32c-3041ed4595ad service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:17.796435 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 842/1665] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:22:17.797401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d39bbd-20e7-4289-b32c-3041ed4595ad service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:17.818288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d39bbd-20e7-4289-b32c-3041ed4595ad service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:17.835151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a87895ee-4ce6-484e-870a-abcdab6607bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc64e6bd0bfd48be9abf02404e59a734', 'request_id': 'req-a87895ee-4ce6-484e-870a-abcdab6607bd'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:17.850056 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a87895ee-4ce6-484e-870a-abcdab6607bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc64e6bd0bfd48be9abf02404e59a734', 'request_id': 'req-a87895ee-4ce6-484e-870a-abcdab6607bd'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:17.858797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a87895ee-4ce6-484e-870a-abcdab6607bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc64e6bd0bfd48be9abf02404e59a734', 'request_id': 'req-a87895ee-4ce6-484e-870a-abcdab6607bd'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:17.885530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a87895ee-4ce6-484e-870a-abcdab6607bd admin admin] The token cache is being invalidate because project cc64e6bd0bfd48be9abf02404e59a734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:17.887517 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a87895ee-4ce6-484e-870a-abcdab6607bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:17.890331 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a87895ee-4ce6-484e-870a-abcdab6607bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:17.892917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 824/1666] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:17 2022] DELETE /identity/v3/projects/cc64e6bd0bfd48be9abf02404e59a734 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:17.904238 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7321f03-ace6-45e8-9465-273b239e24c8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:17.904877 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7321f03-ace6-45e8-9465-273b239e24c8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:17.905257 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7321f03-ace6-45e8-9465-273b239e24c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:17.950248 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e7321f03-ace6-45e8-9465-273b239e24c8 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:17.952728 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e7321f03-ace6-45e8-9465-273b239e24c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:17.979357 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3d4fe-c17a-44d2-a416-613af8460828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:17.980283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3d4fe-c17a-44d2-a416-613af8460828 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:17.981763 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3d4fe-c17a-44d2-a416-613af8460828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:17.983515 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3d4fe-c17a-44d2-a416-613af8460828 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:17.985471 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3d4fe-c17a-44d2-a416-613af8460828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:17.992252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3d4fe-c17a-44d2-a416-613af8460828 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:18.050260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:18.062855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 843/1667] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:17 2022] POST /identity/v3/projects => generated 357 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:18.068278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3d4fe-c17a-44d2-a416-613af8460828 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:18.072196 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e7c3d4fe-c17a-44d2-a416-613af8460828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:18.157486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 825/1668] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:18.168053 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9861e-3948-4fdd-9ca3-8f2e882874c9 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:18.168381 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9861e-3948-4fdd-9ca3-8f2e882874c9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:18.168608 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9861e-3948-4fdd-9ca3-8f2e882874c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:18.217261 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-bcd9861e-3948-4fdd-9ca3-8f2e882874c9 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:18.218500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bcd9861e-3948-4fdd-9ca3-8f2e882874c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:18.279151 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3d4fe-c17a-44d2-a416-613af8460828 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:18.298083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc10e2-4562-436d-99fa-6e72ac5391a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:18.298937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc10e2-4562-436d-99fa-6e72ac5391a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:18.299315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc10e2-4562-436d-99fa-6e72ac5391a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:18.300130 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc10e2-4562-436d-99fa-6e72ac5391a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:18.303475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc10e2-4562-436d-99fa-6e72ac5391a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:18.305869 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc10e2-4562-436d-99fa-6e72ac5391a5 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:18.415573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 844/1669] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:18 2022] POST /identity/v3/users => generated 285 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:18.424731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc10e2-4562-436d-99fa-6e72ac5391a5 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:18.428586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 826/1670] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:18.429646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9861e-3948-4fdd-9ca3-8f2e882874c9 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:18.463210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9861e-3948-4fdd-9ca3-8f2e882874c9 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:18.469426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc10e2-4562-436d-99fa-6e72ac5391a5 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:18.471162 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38155d4e-5a54-4751-9731-4969b410c241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:18.472540 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38155d4e-5a54-4751-9731-4969b410c241 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:18.473158 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38155d4e-5a54-4751-9731-4969b410c241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:18.473654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38155d4e-5a54-4751-9731-4969b410c241 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:18.474722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38155d4e-5a54-4751-9731-4969b410c241 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:18.477994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38155d4e-5a54-4751-9731-4969b410c241 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:18.485690 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38155d4e-5a54-4751-9731-4969b410c241 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:18.510493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3cc20-3d87-4cd4-b469-c84633dcff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:18.517879 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3cc20-3d87-4cd4-b469-c84633dcff00 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:18.525271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3cc20-3d87-4cd4-b469-c84633dcff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:18.525838 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3cc20-3d87-4cd4-b469-c84633dcff00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:18.526760 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3cc20-3d87-4cd4-b469-c84633dcff00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:18.541315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3cc20-3d87-4cd4-b469-c84633dcff00 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:18.561996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 827/1671] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:18.563034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38155d4e-5a54-4751-9731-4969b410c241 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:18.597768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 845/1672] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:18.602029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3cc20-3d87-4cd4-b469-c84633dcff00 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:18.606461 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38155d4e-5a54-4751-9731-4969b410c241 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:18.615326 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48bdb6-7c6d-4137-9f61-c23af64060d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:18.616693 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48bdb6-7c6d-4137-9f61-c23af64060d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:18.617138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48bdb6-7c6d-4137-9f61-c23af64060d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:18.617662 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48bdb6-7c6d-4137-9f61-c23af64060d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:18.618466 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48bdb6-7c6d-4137-9f61-c23af64060d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:18.621110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48bdb6-7c6d-4137-9f61-c23af64060d7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:18.631311 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:18.636868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3cc20-3d87-4cd4-b469-c84633dcff00 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:18.661338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eaa731-b920-45a2-9dbd-217ac0b72c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:18.662179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eaa731-b920-45a2-9dbd-217ac0b72c30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:18.662564 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eaa731-b920-45a2-9dbd-217ac0b72c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:18.662941 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eaa731-b920-45a2-9dbd-217ac0b72c30 admin admin] PATH_INFO: `/v3/projects/6ec7db3bc6fd4cd494124b316a11b31f/users/cc9878080b0745b488341af40d0e8e72/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:18.705865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 828/1673] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:18 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:18.706988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48bdb6-7c6d-4137-9f61-c23af64060d7 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:18.708005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eaa731-b920-45a2-9dbd-217ac0b72c30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ec7db3bc6fd4cd494124b316a11b31f, user_id=cc9878080b0745b488341af40d0e8e72, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:18.712478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eaa731-b920-45a2-9dbd-217ac0b72c30 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:18.752865 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 846/1674] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:18 2022] PUT /identity/v3/projects/6ec7db3bc6fd4cd494124b316a11b31f/users/cc9878080b0745b488341af40d0e8e72/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:18.757253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eaa731-b920-45a2-9dbd-217ac0b72c30 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:18.885722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48bdb6-7c6d-4137-9f61-c23af64060d7 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:18.895599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eaa731-b920-45a2-9dbd-217ac0b72c30 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:18.900425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0567702-807a-4f4a-b065-da9faeb9913e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:18.901040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0567702-807a-4f4a-b065-da9faeb9913e admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:18.901323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0567702-807a-4f4a-b065-da9faeb9913e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:18.901963 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0567702-807a-4f4a-b065-da9faeb9913e admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:18.902229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0567702-807a-4f4a-b065-da9faeb9913e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:18.904204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d37ed-8def-46a6-825d-d535eccad646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:18.904455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0567702-807a-4f4a-b065-da9faeb9913e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:18.905433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d37ed-8def-46a6-825d-d535eccad646 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:18.905924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d37ed-8def-46a6-825d-d535eccad646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:18.906577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d37ed-8def-46a6-825d-d535eccad646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:18.908567 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d37ed-8def-46a6-825d-d535eccad646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:18.911354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d37ed-8def-46a6-825d-d535eccad646 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:18.933567 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 847/1675] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:18.939254 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d37ed-8def-46a6-825d-d535eccad646 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:18.943004 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f03d37ed-8def-46a6-825d-d535eccad646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:18.950946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 829/1676] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:18 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:18.950946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0567702-807a-4f4a-b065-da9faeb9913e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:18.976782 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0567702-807a-4f4a-b065-da9faeb9913e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:18.985755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4e113-d604-4e0e-b8ac-6d7759de25b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:18.986016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d37ed-8def-46a6-825d-d535eccad646 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:18.988217 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4e113-d604-4e0e-b8ac-6d7759de25b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:18.988699 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4e113-d604-4e0e-b8ac-6d7759de25b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:18.989430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4e113-d604-4e0e-b8ac-6d7759de25b4 admin admin] PATH_INFO: `/v3/projects/6ec7db3bc6fd4cd494124b316a11b31f/users/cc9878080b0745b488341af40d0e8e72/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:18.994214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4e113-d604-4e0e-b8ac-6d7759de25b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ec7db3bc6fd4cd494124b316a11b31f, user_id=cc9878080b0745b488341af40d0e8e72, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:18.999158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3e40b-b747-4558-86f5-d69efef78316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:19.001442 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3e40b-b747-4558-86f5-d69efef78316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:19.003208 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3e40b-b747-4558-86f5-d69efef78316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:19.003695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3e40b-b747-4558-86f5-d69efef78316 admin admin] PATH_INFO: `/v3/users/e3bab8e3620642d1a3e8b9d38d4af0ed` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:19.005609 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3e40b-b747-4558-86f5-d69efef78316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3bab8e3620642d1a3e8b9d38d4af0ed)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:19.008051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4e113-d604-4e0e-b8ac-6d7759de25b4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:19.011158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3e40b-b747-4558-86f5-d69efef78316 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:19.044246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 830/1677] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:18 2022] PUT /identity/v3/projects/6ec7db3bc6fd4cd494124b316a11b31f/users/cc9878080b0745b488341af40d0e8e72/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:19.045724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4e113-d604-4e0e-b8ac-6d7759de25b4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:19.126701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:19.131319 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-afa3e40b-b747-4558-86f5-d69efef78316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3bab8e3620642d1a3e8b9d38d4af0ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:19.150998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-afa3e40b-b747-4558-86f5-d69efef78316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3bab8e3620642d1a3e8b9d38d4af0ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:19.151837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-afa3e40b-b747-4558-86f5-d69efef78316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3bab8e3620642d1a3e8b9d38d4af0ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:19.152507 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-afa3e40b-b747-4558-86f5-d69efef78316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3bab8e3620642d1a3e8b9d38d4af0ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:19.168020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-afa3e40b-b747-4558-86f5-d69efef78316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3bab8e3620642d1a3e8b9d38d4af0ed'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:19.208750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4e113-d604-4e0e-b8ac-6d7759de25b4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:19.216332 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 848/1678] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:18 2022] DELETE /identity/v3/users/e3bab8e3620642d1a3e8b9d38d4af0ed => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:19.222886 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd96f0-a866-4361-a95e-5ccfb2e1403f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:19.223144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd96f0-a866-4361-a95e-5ccfb2e1403f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:19.223431 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd96f0-a866-4361-a95e-5ccfb2e1403f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:19.284646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c5dd96f0-a866-4361-a95e-5ccfb2e1403f None None] MFA Rules not processed for user `cc9878080b0745b488341af40d0e8e72`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:19.285996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c5dd96f0-a866-4361-a95e-5ccfb2e1403f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:19.367040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ade4f2-a032-4b72-a226-dc10c4d02dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:19.367824 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ade4f2-a032-4b72-a226-dc10c4d02dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:19.368234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ade4f2-a032-4b72-a226-dc10c4d02dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:19.368671 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ade4f2-a032-4b72-a226-dc10c4d02dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:19.375806 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ade4f2-a032-4b72-a226-dc10c4d02dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:19.378234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ade4f2-a032-4b72-a226-dc10c4d02dc1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:19.401738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 831/1679] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:19.403320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf29a8e-f54c-4144-943e-f8ccd7025463 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:19.405033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf29a8e-f54c-4144-943e-f8ccd7025463 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:19.406467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf29a8e-f54c-4144-943e-f8ccd7025463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:19.442943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:19.470399 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 849/1680] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:19.470486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ddf29a8e-f54c-4144-943e-f8ccd7025463 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:19.470486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ddf29a8e-f54c-4144-943e-f8ccd7025463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:19.470779 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd96f0-a866-4361-a95e-5ccfb2e1403f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:19.473306 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c5dd96f0-a866-4361-a95e-5ccfb2e1403f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:19.494926 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd96f0-a866-4361-a95e-5ccfb2e1403f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:19.498340 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 832/1681] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:19.501516 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf29a8e-f54c-4144-943e-f8ccd7025463 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:19.503132 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c82ed-2806-473d-8606-42dd257da776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:19.503910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ddf29a8e-f54c-4144-943e-f8ccd7025463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:19.531309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c82ed-2806-473d-8606-42dd257da776 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:19.532079 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c82ed-2806-473d-8606-42dd257da776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:19.534472 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c82ed-2806-473d-8606-42dd257da776 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:19.536124 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c82ed-2806-473d-8606-42dd257da776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:19.543399 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c82ed-2806-473d-8606-42dd257da776 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:19.580165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 850/1682] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:19.582097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c82ed-2806-473d-8606-42dd257da776 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:19.625634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c82ed-2806-473d-8606-42dd257da776 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:19.636561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af67e8-4b42-46a1-b594-24180f69ef8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:19.637637 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af67e8-4b42-46a1-b594-24180f69ef8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:19.640452 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af67e8-4b42-46a1-b594-24180f69ef8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:19.640853 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af67e8-4b42-46a1-b594-24180f69ef8a admin admin] PATH_INFO: `/v3/projects/d7ec442aa479492783daf603f9a284ce/users/15ba03725c8449fb9c9eb053ea6e4606/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:19.674350 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf29a8e-f54c-4144-943e-f8ccd7025463 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:19.682580 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c71b6-c22b-4279-a036-a242084a84b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:19.683490 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c71b6-c22b-4279-a036-a242084a84b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:19.684393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c71b6-c22b-4279-a036-a242084a84b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:19.685913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af67e8-4b42-46a1-b594-24180f69ef8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ec442aa479492783daf603f9a284ce, user_id=15ba03725c8449fb9c9eb053ea6e4606, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:19.686300 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c71b6-c22b-4279-a036-a242084a84b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:19.692949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af67e8-4b42-46a1-b594-24180f69ef8a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:19.706685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c71b6-c22b-4279-a036-a242084a84b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:19.718910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c71b6-c22b-4279-a036-a242084a84b8 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:19.735822 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 851/1683] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:19 2022] PUT /identity/v3/projects/d7ec442aa479492783daf603f9a284ce/users/15ba03725c8449fb9c9eb053ea6e4606/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:19.737539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f5b02-f8a3-4b88-8ec9-39d3ee623432 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:19.737831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f5b02-f8a3-4b88-8ec9-39d3ee623432 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:19.738100 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f5b02-f8a3-4b88-8ec9-39d3ee623432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:19.744320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 833/1684] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:19.757174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c71b6-c22b-4279-a036-a242084a84b8 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:19.765632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4d3c71b6-c22b-4279-a036-a242084a84b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:19.820519 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ce1f5b02-f8a3-4b88-8ec9-39d3ee623432 None None] MFA Rules not processed for user `15ba03725c8449fb9c9eb053ea6e4606`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:19.822260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ce1f5b02-f8a3-4b88-8ec9-39d3ee623432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:19.965551 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c71b6-c22b-4279-a036-a242084a84b8 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:19.977715 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 852/1685] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:19.977715 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f5b02-f8a3-4b88-8ec9-39d3ee623432 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:19.992882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:20.101208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480d193-7ce7-4ba5-99a3-ff9988c4f0ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:20.101848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480d193-7ce7-4ba5-99a3-ff9988c4f0ba service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:20.102124 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480d193-7ce7-4ba5-99a3-ff9988c4f0ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:20.102399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480d193-7ce7-4ba5-99a3-ff9988c4f0ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:20.112106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f5b02-f8a3-4b88-8ec9-39d3ee623432 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:20.123648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63171f71-c36a-40ea-9669-de812aff3f19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:20.124575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63171f71-c36a-40ea-9669-de812aff3f19 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:20.124860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63171f71-c36a-40ea-9669-de812aff3f19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:20.125130 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63171f71-c36a-40ea-9669-de812aff3f19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:20.131551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-63171f71-c36a-40ea-9669-de812aff3f19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:20.225766 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9480d193-7ce7-4ba5-99a3-ff9988c4f0ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:20.229013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9480d193-7ce7-4ba5-99a3-ff9988c4f0ba service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:20.256532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 834/1686] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:20.258596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480d193-7ce7-4ba5-99a3-ff9988c4f0ba service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:20.274693 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63171f71-c36a-40ea-9669-de812aff3f19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:20.280785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63171f71-c36a-40ea-9669-de812aff3f19 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:20.282821 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480d193-7ce7-4ba5-99a3-ff9988c4f0ba service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:20.320129 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 853/1687] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:19 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:20.320342 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d06e7-15a3-4c7e-b1d6-1cb4181ecfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:20.320954 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d06e7-15a3-4c7e-b1d6-1cb4181ecfe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:20.321240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d06e7-15a3-4c7e-b1d6-1cb4181ecfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:20.321516 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d06e7-15a3-4c7e-b1d6-1cb4181ecfe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:20.322604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d06e7-15a3-4c7e-b1d6-1cb4181ecfe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:20.324110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d06e7-15a3-4c7e-b1d6-1cb4181ecfe6 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:20.340932 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:20.396881 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63171f71-c36a-40ea-9669-de812aff3f19 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:20.420323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 835/1688] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:20 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:20.428481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63171f71-c36a-40ea-9669-de812aff3f19 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:20.572360 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fc6ba-2603-42fa-b1fd-eaa107746041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:20.573555 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fc6ba-2603-42fa-b1fd-eaa107746041 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:20.573830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fc6ba-2603-42fa-b1fd-eaa107746041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:20.574111 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fc6ba-2603-42fa-b1fd-eaa107746041 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:20.576118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3fc6ba-2603-42fa-b1fd-eaa107746041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:20.583551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3fc6ba-2603-42fa-b1fd-eaa107746041 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:20.681342 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d06e7-15a3-4c7e-b1d6-1cb4181ecfe6 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:20.694986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 854/1689] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:20 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:20.709304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d06e7-15a3-4c7e-b1d6-1cb4181ecfe6 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:20.720390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fc6ba-2603-42fa-b1fd-eaa107746041 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:20.724138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7c3fc6ba-2603-42fa-b1fd-eaa107746041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:20.735497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714b0a1-ffcd-43b9-a65a-a19c2725d762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:20.737893 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714b0a1-ffcd-43b9-a65a-a19c2725d762 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:20.757511 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714b0a1-ffcd-43b9-a65a-a19c2725d762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:20.758256 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714b0a1-ffcd-43b9-a65a-a19c2725d762 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:20.761296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714b0a1-ffcd-43b9-a65a-a19c2725d762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:20.771114 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714b0a1-ffcd-43b9-a65a-a19c2725d762 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:20.778516 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fc6ba-2603-42fa-b1fd-eaa107746041 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:20.793256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d63380-3f70-4af1-9124-d3d11fdfae62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:20.795279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d63380-3f70-4af1-9124-d3d11fdfae62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:20.795590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d63380-3f70-4af1-9124-d3d11fdfae62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:20.795866 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d63380-3f70-4af1-9124-d3d11fdfae62 admin admin] PATH_INFO: `/v3/projects/a214702b5d894605a09e9321e280a5b1` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:20.799785 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 836/1690] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:20.805094 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d63380-3f70-4af1-9124-d3d11fdfae62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a214702b5d894605a09e9321e280a5b1)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:20.811001 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714b0a1-ffcd-43b9-a65a-a19c2725d762 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:20.811171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d63380-3f70-4af1-9124-d3d11fdfae62 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:20.839984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714b0a1-ffcd-43b9-a65a-a19c2725d762 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:20.852861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398b47b-f374-46a6-a224-6834b8ba7d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:20.854513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398b47b-f374-46a6-a224-6834b8ba7d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:20.855758 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398b47b-f374-46a6-a224-6834b8ba7d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:20.856036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398b47b-f374-46a6-a224-6834b8ba7d49 admin admin] PATH_INFO: `/v3/projects/41807e498b3947aca80280634840ef6b/users/5af0791c13bf4716871968a725c42de2/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:20.917478 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398b47b-f374-46a6-a224-6834b8ba7d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41807e498b3947aca80280634840ef6b, user_id=5af0791c13bf4716871968a725c42de2, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:20.921843 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398b47b-f374-46a6-a224-6834b8ba7d49 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:20.962831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 837/1691] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:20 2022] PUT /identity/v3/projects/41807e498b3947aca80280634840ef6b/users/5af0791c13bf4716871968a725c42de2/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:20.964781 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:20.967262 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398b47b-f374-46a6-a224-6834b8ba7d49 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:20.983743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-45d63380-3f70-4af1-9124-d3d11fdfae62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a214702b5d894605a09e9321e280a5b1', 'request_id': 'req-45d63380-3f70-4af1-9124-d3d11fdfae62'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:21.005478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-45d63380-3f70-4af1-9124-d3d11fdfae62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a214702b5d894605a09e9321e280a5b1', 'request_id': 'req-45d63380-3f70-4af1-9124-d3d11fdfae62'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:21.020333 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-45d63380-3f70-4af1-9124-d3d11fdfae62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a214702b5d894605a09e9321e280a5b1', 'request_id': 'req-45d63380-3f70-4af1-9124-d3d11fdfae62'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:21.049449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-45d63380-3f70-4af1-9124-d3d11fdfae62 admin admin] The token cache is being invalidate because project a214702b5d894605a09e9321e280a5b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:21.049754 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-45d63380-3f70-4af1-9124-d3d11fdfae62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:21.054373 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-45d63380-3f70-4af1-9124-d3d11fdfae62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:21.062322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 855/1692] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:20 2022] DELETE /identity/v3/projects/a214702b5d894605a09e9321e280a5b1 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:21.064699 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cad00-c9c4-46b4-899a-799e1c64935e None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:21.064986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cad00-c9c4-46b4-899a-799e1c64935e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:21.065261 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cad00-c9c4-46b4-899a-799e1c64935e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:21.145912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-9c0cad00-c9c4-46b4-899a-799e1c64935e None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:21.147371 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9c0cad00-c9c4-46b4-899a-799e1c64935e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:21.184443 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398b47b-f374-46a6-a224-6834b8ba7d49 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:21.190399 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 856/1693] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:21.195593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369e5f5-d666-4fac-8936-3463d86da78a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:21.195857 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369e5f5-d666-4fac-8936-3463d86da78a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:21.196187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369e5f5-d666-4fac-8936-3463d86da78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:21.206513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded6a1a-ab7c-4876-a0c7-7e6d51236a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:21.207146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded6a1a-ab7c-4876-a0c7-7e6d51236a70 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:21.207431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded6a1a-ab7c-4876-a0c7-7e6d51236a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:21.207794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded6a1a-ab7c-4876-a0c7-7e6d51236a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:21.208545 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded6a1a-ab7c-4876-a0c7-7e6d51236a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:21.210185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded6a1a-ab7c-4876-a0c7-7e6d51236a70 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:21.243839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 838/1694] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:21.246606 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded6a1a-ab7c-4876-a0c7-7e6d51236a70 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:21.249825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7ded6a1a-ab7c-4876-a0c7-7e6d51236a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:21.256545 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3369e5f5-d666-4fac-8936-3463d86da78a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:21.259051 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3369e5f5-d666-4fac-8936-3463d86da78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:21.275565 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded6a1a-ab7c-4876-a0c7-7e6d51236a70 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:21.288133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 857/1695] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:21.289533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d27ab-2f1e-4fb0-911e-fc7906717fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:21.290238 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d27ab-2f1e-4fb0-911e-fc7906717fd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:21.290522 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d27ab-2f1e-4fb0-911e-fc7906717fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:21.290797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d27ab-2f1e-4fb0-911e-fc7906717fd7 admin admin] PATH_INFO: `/v3/projects/41807e498b3947aca80280634840ef6b/users/5af0791c13bf4716871968a725c42de2/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:21.296483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369e5f5-d666-4fac-8936-3463d86da78a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:21.297844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d27ab-2f1e-4fb0-911e-fc7906717fd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41807e498b3947aca80280634840ef6b, user_id=5af0791c13bf4716871968a725c42de2, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:21.300591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d27ab-2f1e-4fb0-911e-fc7906717fd7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:21.316044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369e5f5-d666-4fac-8936-3463d86da78a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:21.331772 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 839/1696] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:21 2022] PUT /identity/v3/projects/41807e498b3947aca80280634840ef6b/users/5af0791c13bf4716871968a725c42de2/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:21.344387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c65238-a1ea-48d9-91ac-add5f0a30498 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:21.344689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c65238-a1ea-48d9-91ac-add5f0a30498 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:21.345025 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c65238-a1ea-48d9-91ac-add5f0a30498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:21.349732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26dfb4-e512-47f2-8d6b-34f68f94521b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:21.350371 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26dfb4-e512-47f2-8d6b-34f68f94521b admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:21.350706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26dfb4-e512-47f2-8d6b-34f68f94521b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:21.351091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26dfb4-e512-47f2-8d6b-34f68f94521b admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:21.352014 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26dfb4-e512-47f2-8d6b-34f68f94521b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:21.352149 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26dfb4-e512-47f2-8d6b-34f68f94521b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:21.354211 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e26dfb4-e512-47f2-8d6b-34f68f94521b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:21.382540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 858/1697] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:21.386272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26dfb4-e512-47f2-8d6b-34f68f94521b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:21.404422 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e26dfb4-e512-47f2-8d6b-34f68f94521b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:21.408840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-87c65238-a1ea-48d9-91ac-add5f0a30498 None None] MFA Rules not processed for user `5af0791c13bf4716871968a725c42de2`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:21.410073 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-87c65238-a1ea-48d9-91ac-add5f0a30498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:21.544141 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de5062-9f8c-4fd8-b676-ea035ae7197f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:21.546246 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de5062-9f8c-4fd8-b676-ea035ae7197f admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:21.547393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de5062-9f8c-4fd8-b676-ea035ae7197f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:21.547861 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de5062-9f8c-4fd8-b676-ea035ae7197f admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:21.549979 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de5062-9f8c-4fd8-b676-ea035ae7197f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:21.552302 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de5062-9f8c-4fd8-b676-ea035ae7197f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:21.589476 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 840/1698] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:22:21.612840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c65238-a1ea-48d9-91ac-add5f0a30498 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:21.615120 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-87c65238-a1ea-48d9-91ac-add5f0a30498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:21.634850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 859/1699] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:21 2022] POST /identity/v3/projects => generated 381 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:21.639369 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de5062-9f8c-4fd8-b676-ea035ae7197f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:21.707757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:21.802541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c65238-a1ea-48d9-91ac-add5f0a30498 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:21.838382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de5062-9f8c-4fd8-b676-ea035ae7197f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:21.854159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7628d1f-5ac7-42d6-8038-fafde343a088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:21.854789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7628d1f-5ac7-42d6-8038-fafde343a088 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:21.855068 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7628d1f-5ac7-42d6-8038-fafde343a088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:21.855339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7628d1f-5ac7-42d6-8038-fafde343a088 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:21.862332 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7628d1f-5ac7-42d6-8038-fafde343a088 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:21.872164 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7628d1f-5ac7-42d6-8038-fafde343a088 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:21.940685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 860/1700] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:21 2022] POST /identity/v3/users => generated 291 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:21.941234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7628d1f-5ac7-42d6-8038-fafde343a088 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:21.944755 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b7628d1f-5ac7-42d6-8038-fafde343a088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:21.955623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fc5b5-7333-4e76-8f76-8fbdd8536ca6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:21.956815 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fc5b5-7333-4e76-8f76-8fbdd8536ca6 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:21.957358 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fc5b5-7333-4e76-8f76-8fbdd8536ca6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:21.957892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fc5b5-7333-4e76-8f76-8fbdd8536ca6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:21.994127 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7628d1f-5ac7-42d6-8038-fafde343a088 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:22.010818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d95175-6ecb-4183-91da-18877a523e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:22.011924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d95175-6ecb-4183-91da-18877a523e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:22.014656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d95175-6ecb-4183-91da-18877a523e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:22.015310 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d95175-6ecb-4183-91da-18877a523e79 admin admin] PATH_INFO: `/v3/projects/a4c444b37c884b8aad96c00dda9615b2` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:22.017256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d95175-6ecb-4183-91da-18877a523e79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4c444b37c884b8aad96c00dda9615b2)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:22.023819 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d95175-6ecb-4183-91da-18877a523e79 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:22.060905 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:22.111640 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fc5b5-7333-4e76-8f76-8fbdd8536ca6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:22.116680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fc5b5-7333-4e76-8f76-8fbdd8536ca6 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:22.145282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-40d95175-6ecb-4183-91da-18877a523e79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4c444b37c884b8aad96c00dda9615b2', 'request_id': 'req-40d95175-6ecb-4183-91da-18877a523e79'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:22.147621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 841/1701] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:22:22.149040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc8042-8da5-435a-be00-37c72055a733 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:22.150682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc8042-8da5-435a-be00-37c72055a733 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:22.151416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc8042-8da5-435a-be00-37c72055a733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:22.184963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-40d95175-6ecb-4183-91da-18877a523e79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4c444b37c884b8aad96c00dda9615b2', 'request_id': 'req-40d95175-6ecb-4183-91da-18877a523e79'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:22.196029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-40d95175-6ecb-4183-91da-18877a523e79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4c444b37c884b8aad96c00dda9615b2', 'request_id': 'req-40d95175-6ecb-4183-91da-18877a523e79'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:22.210921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-06cc8042-8da5-435a-be00-37c72055a733 None None] MFA Rules not processed for user `1f4cbe342b2c4378b3c0ab2f1cd55260`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:22.212929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-06cc8042-8da5-435a-be00-37c72055a733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:22.250207 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-40d95175-6ecb-4183-91da-18877a523e79 admin admin] The token cache is being invalidate because project a4c444b37c884b8aad96c00dda9615b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:22.250663 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-40d95175-6ecb-4183-91da-18877a523e79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:22.253781 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-40d95175-6ecb-4183-91da-18877a523e79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:22.258418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 861/1702] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:21 2022] DELETE /identity/v3/projects/a4c444b37c884b8aad96c00dda9615b2 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:22.259719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d95175-6ecb-4183-91da-18877a523e79 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:22.263817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-40d95175-6ecb-4183-91da-18877a523e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:22.350387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 842/1703] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:22.351324 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc8042-8da5-435a-be00-37c72055a733 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:22.352743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-06cc8042-8da5-435a-be00-37c72055a733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:22.394844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:22.407161 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d95175-6ecb-4183-91da-18877a523e79 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:22.419041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600e62c-ec66-40fc-896a-e0a638bd8c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:22.420396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600e62c-ec66-40fc-896a-e0a638bd8c4c admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:22.421081 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600e62c-ec66-40fc-896a-e0a638bd8c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:22.421528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600e62c-ec66-40fc-896a-e0a638bd8c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:22.422573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600e62c-ec66-40fc-896a-e0a638bd8c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:22.425147 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600e62c-ec66-40fc-896a-e0a638bd8c4c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:22.442408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 862/1704] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:22.443629 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600e62c-ec66-40fc-896a-e0a638bd8c4c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:22.445820 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5600e62c-ec66-40fc-896a-e0a638bd8c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:22.484955 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc8042-8da5-435a-be00-37c72055a733 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:22.492113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed2468-7b2d-446c-9940-5dbc76f55450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:22.492829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed2468-7b2d-446c-9940-5dbc76f55450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:22.493118 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed2468-7b2d-446c-9940-5dbc76f55450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:22.493874 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed2468-7b2d-446c-9940-5dbc76f55450 admin admin] PATH_INFO: `/v3/users/810939fd49084e9ba9b12e933a386178` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:22.494724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed2468-7b2d-446c-9940-5dbc76f55450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=810939fd49084e9ba9b12e933a386178)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:22.497108 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed2468-7b2d-446c-9940-5dbc76f55450 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:22.543656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600e62c-ec66-40fc-896a-e0a638bd8c4c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:22.550070 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440be742-278d-45d6-8725-b46d4e526f09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:22.551705 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440be742-278d-45d6-8725-b46d4e526f09 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:22.552393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440be742-278d-45d6-8725-b46d4e526f09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:22.552686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440be742-278d-45d6-8725-b46d4e526f09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:22.554722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-440be742-278d-45d6-8725-b46d4e526f09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:22.577615 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-09ed2468-7b2d-446c-9940-5dbc76f55450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '810939fd49084e9ba9b12e933a386178'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:22.595254 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-09ed2468-7b2d-446c-9940-5dbc76f55450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '810939fd49084e9ba9b12e933a386178'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:22.596841 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-09ed2468-7b2d-446c-9940-5dbc76f55450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '810939fd49084e9ba9b12e933a386178'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:22.597491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-09ed2468-7b2d-446c-9940-5dbc76f55450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '810939fd49084e9ba9b12e933a386178'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:22.618181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-09ed2468-7b2d-446c-9940-5dbc76f55450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '810939fd49084e9ba9b12e933a386178'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:22.647610 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 843/1705] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:22 2022] DELETE /identity/v3/users/810939fd49084e9ba9b12e933a386178 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:22.649046 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4742d-bcb5-411c-b380-4e7ea0873c76 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:22.649344 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4742d-bcb5-411c-b380-4e7ea0873c76 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:22.649625 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4742d-bcb5-411c-b380-4e7ea0873c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:22.672709 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:22.692676 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-0de4742d-bcb5-411c-b380-4e7ea0873c76 None None] MFA Rules not processed for user `1f4cbe342b2c4378b3c0ab2f1cd55260`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:22.694048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0de4742d-bcb5-411c-b380-4e7ea0873c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:22.702722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440be742-278d-45d6-8725-b46d4e526f09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:22.704304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440be742-278d-45d6-8725-b46d4e526f09 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:22.728729 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 863/1706] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:22.742493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9aec7-57ef-4bc7-8d87-dc3af7921bed None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:22.742792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9aec7-57ef-4bc7-8d87-dc3af7921bed None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:22.743070 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9aec7-57ef-4bc7-8d87-dc3af7921bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:22.824306 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 844/1707] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:22.825753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4742d-bcb5-411c-b380-4e7ea0873c76 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:22.831016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0de4742d-bcb5-411c-b380-4e7ea0873c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:22.909183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6ae9aec7-57ef-4bc7-8d87-dc3af7921bed None None] MFA Rules not processed for user `3066c1c9c79942c289b733d886d9fd0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:22.915395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6ae9aec7-57ef-4bc7-8d87-dc3af7921bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:22.941866 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:22.978048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4742d-bcb5-411c-b380-4e7ea0873c76 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:22.991977 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304356f-2c7d-4a29-a32e-4302412e9705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:22.992686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304356f-2c7d-4a29-a32e-4302412e9705 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:22.993784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304356f-2c7d-4a29-a32e-4302412e9705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:22.994079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304356f-2c7d-4a29-a32e-4302412e9705 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:22.994805 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3304356f-2c7d-4a29-a32e-4302412e9705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:22.996733 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3304356f-2c7d-4a29-a32e-4302412e9705 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:23.014911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 845/1708] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:23.014911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304356f-2c7d-4a29-a32e-4302412e9705 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:23.018750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3304356f-2c7d-4a29-a32e-4302412e9705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:23.166768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 864/1709] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:23.169269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9aec7-57ef-4bc7-8d87-dc3af7921bed None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:23.206719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9aec7-57ef-4bc7-8d87-dc3af7921bed None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:23.220153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986023c-3f3b-487a-a285-1dce02688a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:23.223157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986023c-3f3b-487a-a285-1dce02688a78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:23.223477 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986023c-3f3b-487a-a285-1dce02688a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:23.223793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986023c-3f3b-487a-a285-1dce02688a78 admin admin] PATH_INFO: `/v3/projects/99447455b6974f4888380e172e16c46e/users/c90ad34bac7a433681256368c36126da/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:23.228368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304356f-2c7d-4a29-a32e-4302412e9705 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:23.237246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac7e9c-f5d0-4da3-82da-e28e5bfd6407 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:23.238041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac7e9c-f5d0-4da3-82da-e28e5bfd6407 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:23.238396 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac7e9c-f5d0-4da3-82da-e28e5bfd6407 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:23.238746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac7e9c-f5d0-4da3-82da-e28e5bfd6407 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:23.245135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c2ac7e9c-f5d0-4da3-82da-e28e5bfd6407 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:23.263283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986023c-3f3b-487a-a285-1dce02688a78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99447455b6974f4888380e172e16c46e, user_id=c90ad34bac7a433681256368c36126da, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:23.263469 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac7e9c-f5d0-4da3-82da-e28e5bfd6407 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:23.265884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac7e9c-f5d0-4da3-82da-e28e5bfd6407 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:23.269401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986023c-3f3b-487a-a285-1dce02688a78 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:23.296207 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 846/1710] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:23.297986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac7e9c-f5d0-4da3-82da-e28e5bfd6407 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:23.304213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c2ac7e9c-f5d0-4da3-82da-e28e5bfd6407 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:23.313551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 865/1711] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:23 2022] PUT /identity/v3/projects/99447455b6974f4888380e172e16c46e/users/c90ad34bac7a433681256368c36126da/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:23.315035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b640ca-c208-41c2-b0ff-4baf22add653 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:23.315451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b640ca-c208-41c2-b0ff-4baf22add653 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:23.315826 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b640ca-c208-41c2-b0ff-4baf22add653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:23.350793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac7e9c-f5d0-4da3-82da-e28e5bfd6407 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:23.397281 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b5b640ca-c208-41c2-b0ff-4baf22add653 None None] MFA Rules not processed for user `c90ad34bac7a433681256368c36126da`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:23.399029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b5b640ca-c208-41c2-b0ff-4baf22add653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:23.515155 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4f595-9240-4f5b-abf0-d7871bc3f2f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:23.515939 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4f595-9240-4f5b-abf0-d7871bc3f2f5 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:23.516286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4f595-9240-4f5b-abf0-d7871bc3f2f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:23.516629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4f595-9240-4f5b-abf0-d7871bc3f2f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:23.609047 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 866/1712] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:23.614699 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b640ca-c208-41c2-b0ff-4baf22add653 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:23.668914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb4f595-9240-4f5b-abf0-d7871bc3f2f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:23.670884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb4f595-9240-4f5b-abf0-d7871bc3f2f5 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:23.679726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:23.695880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 847/1713] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:23.710703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:23.742228 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b640ca-c208-41c2-b0ff-4baf22add653 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:23.754716 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7c95c-20dd-4c06-807b-5175fa2bf173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:23.755403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7c95c-20dd-4c06-807b-5175fa2bf173 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:23.755884 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7c95c-20dd-4c06-807b-5175fa2bf173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:23.756158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7c95c-20dd-4c06-807b-5175fa2bf173 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:23.757878 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7c95c-20dd-4c06-807b-5175fa2bf173 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:23.763030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7c95c-20dd-4c06-807b-5175fa2bf173 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:23.844033 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 867/1714] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:23 2022] POST /identity/v3/projects => generated 381 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:23.848070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4f595-9240-4f5b-abf0-d7871bc3f2f5 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:23.878263 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4f595-9240-4f5b-abf0-d7871bc3f2f5 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:24.009624 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a8526-ef45-453c-9742-979a15ae7db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:24.010769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a8526-ef45-453c-9742-979a15ae7db7 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:24.011400 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a8526-ef45-453c-9742-979a15ae7db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:24.013383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a8526-ef45-453c-9742-979a15ae7db7 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:24.016215 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a8526-ef45-453c-9742-979a15ae7db7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:24.018971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a8526-ef45-453c-9742-979a15ae7db7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:24.087947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7c95c-20dd-4c06-807b-5175fa2bf173 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:24.090259 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-43a7c95c-20dd-4c06-807b-5175fa2bf173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:24.094680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 848/1715] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:23 2022] POST /identity/v3/users => generated 297 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:24.099581 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a8526-ef45-453c-9742-979a15ae7db7 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:24.128172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7c95c-20dd-4c06-807b-5175fa2bf173 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:24.133932 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a8526-ef45-453c-9742-979a15ae7db7 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:24.137525 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:24.139319 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:24.139638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:24.139901 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931 admin admin] PATH_INFO: `/v3/projects/e3d5d86f3ba34f3ea4fec1821103e1fc` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:24.148215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d5d86f3ba34f3ea4fec1821103e1fc)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:24.148475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf7b42-f8b2-4f7a-bd5d-f8c01bb3db74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:24.150123 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf7b42-f8b2-4f7a-bd5d-f8c01bb3db74 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:24.150420 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf7b42-f8b2-4f7a-bd5d-f8c01bb3db74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:24.150525 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:24.150685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf7b42-f8b2-4f7a-bd5d-f8c01bb3db74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:24.154648 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf7b42-f8b2-4f7a-bd5d-f8c01bb3db74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:24.159219 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf7b42-f8b2-4f7a-bd5d-f8c01bb3db74 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:24.196625 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 849/1716] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:24.202023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf7b42-f8b2-4f7a-bd5d-f8c01bb3db74 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:24.252428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf7b42-f8b2-4f7a-bd5d-f8c01bb3db74 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:24.276194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d5d86f3ba34f3ea4fec1821103e1fc', 'request_id': 'req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:24.299814 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d5d86f3ba34f3ea4fec1821103e1fc', 'request_id': 'req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:24.316714 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d5d86f3ba34f3ea4fec1821103e1fc', 'request_id': 'req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:24.348855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931 admin admin] The token cache is being invalidate because project e3d5d86f3ba34f3ea4fec1821103e1fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:24.350444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:24.351369 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:24.362974 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 868/1717] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:24 2022] DELETE /identity/v3/projects/e3d5d86f3ba34f3ea4fec1821103e1fc => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:24.364027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:24.369143 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:24.387576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0bd3a-6089-442f-86df-68a98f065535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:24.388235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0bd3a-6089-442f-86df-68a98f065535 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:24.388518 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0bd3a-6089-442f-86df-68a98f065535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:24.388975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0bd3a-6089-442f-86df-68a98f065535 admin admin] PATH_INFO: `/v3/projects/4f146b5cf187406aa04b767aac3ce891/users/fda45ac8ce02440ba5c6002bac3cd11d/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:24.410182 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0bd3a-6089-442f-86df-68a98f065535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f146b5cf187406aa04b767aac3ce891, user_id=fda45ac8ce02440ba5c6002bac3cd11d, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:24.414827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0bd3a-6089-442f-86df-68a98f065535 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:24.447648 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 850/1718] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:24 2022] PUT /identity/v3/projects/4f146b5cf187406aa04b767aac3ce891/users/fda45ac8ce02440ba5c6002bac3cd11d/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:24.449301 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c976e-d38e-407e-b3ff-c9d3a5697b64 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:24.449659 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c976e-d38e-407e-b3ff-c9d3a5697b64 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:24.450568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c976e-d38e-407e-b3ff-c9d3a5697b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:24.517398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fb5cd-ec49-4b5f-afcf-31a50d360931 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:24.518211 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e24c976e-d38e-407e-b3ff-c9d3a5697b64 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:24.521604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e24c976e-d38e-407e-b3ff-c9d3a5697b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:24.639611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cc25e-85f5-4164-9e25-a7a06043d32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:24.640710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cc25e-85f5-4164-9e25-a7a06043d32d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:24.642020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cc25e-85f5-4164-9e25-a7a06043d32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:24.643059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cc25e-85f5-4164-9e25-a7a06043d32d admin admin] PATH_INFO: `/v3/users/78c2026a886744e1bcaab9420d14dc88` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:24.644978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cc25e-85f5-4164-9e25-a7a06043d32d admin admin] RBAC: Authorizing `identity:delete_user(user_id=78c2026a886744e1bcaab9420d14dc88)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:24.648065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cc25e-85f5-4164-9e25-a7a06043d32d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:24.651296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:24.684684 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:24.708198 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 851/1719] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:24.710614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c976e-d38e-407e-b3ff-c9d3a5697b64 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:24.712277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e24c976e-d38e-407e-b3ff-c9d3a5697b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:24.744106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6c7cc25e-85f5-4164-9e25-a7a06043d32d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78c2026a886744e1bcaab9420d14dc88'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:24.748990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c976e-d38e-407e-b3ff-c9d3a5697b64 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:24.773776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bcf31-4471-40a5-a3ee-e1b743ea4310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:24.774195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bcf31-4471-40a5-a3ee-e1b743ea4310 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:24.774487 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bcf31-4471-40a5-a3ee-e1b743ea4310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:24.775048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6c7cc25e-85f5-4164-9e25-a7a06043d32d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78c2026a886744e1bcaab9420d14dc88'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:24.776030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6c7cc25e-85f5-4164-9e25-a7a06043d32d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78c2026a886744e1bcaab9420d14dc88'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:24.776850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6c7cc25e-85f5-4164-9e25-a7a06043d32d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78c2026a886744e1bcaab9420d14dc88'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:24.778065 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bcf31-4471-40a5-a3ee-e1b743ea4310 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:24.783695 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bcf31-4471-40a5-a3ee-e1b743ea4310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:24.788905 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bcf31-4471-40a5-a3ee-e1b743ea4310 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:24.797100 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6c7cc25e-85f5-4164-9e25-a7a06043d32d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78c2026a886744e1bcaab9420d14dc88'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:24.828076 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 852/1720] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:24.829457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea205d-89c7-4155-8fef-92aee66207b1 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:24.829747 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea205d-89c7-4155-8fef-92aee66207b1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:24.839558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 869/1721] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:24 2022] DELETE /identity/v3/users/78c2026a886744e1bcaab9420d14dc88 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:24.841807 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea205d-89c7-4155-8fef-92aee66207b1 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:24.844024 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c3aa0-6962-411f-8b27-072c363bbdeb None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:24.844577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c3aa0-6962-411f-8b27-072c363bbdeb None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:24.845068 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c3aa0-6962-411f-8b27-072c363bbdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:24.854494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 853/1722] 10.222.0.33 () {54 vars in 912 bytes} [Thu Oct 6 12:22:24 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:22:24.855466 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea205d-89c7-4155-8fef-92aee66207b1 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:24.865454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-71ea205d-89c7-4155-8fef-92aee66207b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:24.908001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-975c3aa0-6962-411f-8b27-072c363bbdeb None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:24.909442 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-975c3aa0-6962-411f-8b27-072c363bbdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:25.054858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea205d-89c7-4155-8fef-92aee66207b1 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:25.062784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb84ef2-b92b-4ccf-b80f-d8ae2235909f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:25.063869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb84ef2-b92b-4ccf-b80f-d8ae2235909f admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:25.087248 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb84ef2-b92b-4ccf-b80f-d8ae2235909f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:25.087660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb84ef2-b92b-4ccf-b80f-d8ae2235909f admin admin] PATH_INFO: `/v3/projects/4f146b5cf187406aa04b767aac3ce891/users/fda45ac8ce02440ba5c6002bac3cd11d/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:25.097356 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb84ef2-b92b-4ccf-b80f-d8ae2235909f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f146b5cf187406aa04b767aac3ce891, user_id=fda45ac8ce02440ba5c6002bac3cd11d, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:25.115888 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb84ef2-b92b-4ccf-b80f-d8ae2235909f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:25.150496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 854/1723] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:24 2022] PUT /identity/v3/projects/4f146b5cf187406aa04b767aac3ce891/users/fda45ac8ce02440ba5c6002bac3cd11d/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:25.157866 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd69cc-7bab-413c-9550-dc2b06b0dc53 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:25.158173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd69cc-7bab-413c-9550-dc2b06b0dc53 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:25.158592 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd69cc-7bab-413c-9550-dc2b06b0dc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:25.207076 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 870/1724] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:25.221326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f4326-4fe7-40c7-8979-0d9513aee6fa None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:25.225798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f4326-4fe7-40c7-8979-0d9513aee6fa None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:25.226071 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f4326-4fe7-40c7-8979-0d9513aee6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:25.257706 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-59cd69cc-7bab-413c-9550-dc2b06b0dc53 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:25.258980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-59cd69cc-7bab-413c-9550-dc2b06b0dc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:25.391169 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-281f4326-4fe7-40c7-8979-0d9513aee6fa None None] MFA Rules not processed for user `2e71f9c1626f4f58916a5dd5c5987ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:25.392510 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-281f4326-4fe7-40c7-8979-0d9513aee6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:25.493637 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 855/1725] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:25.496232 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a534d5-37cd-4f92-aa9b-980d71cc3206 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:25.496518 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a534d5-37cd-4f92-aa9b-980d71cc3206 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:25.498291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 871/1726] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Oct 6 12:22:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:25.498948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a534d5-37cd-4f92-aa9b-980d71cc3206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:25.501244 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f4326-4fe7-40c7-8979-0d9513aee6fa None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:25.533270 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f4326-4fe7-40c7-8979-0d9513aee6fa None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:25.550896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba74db2-d4e6-41af-b630-9b6796d9619e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:25.551543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba74db2-d4e6-41af-b630-9b6796d9619e admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:25.551827 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba74db2-d4e6-41af-b630-9b6796d9619e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:25.553149 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba74db2-d4e6-41af-b630-9b6796d9619e admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:25.553986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba74db2-d4e6-41af-b630-9b6796d9619e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:25.554155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba74db2-d4e6-41af-b630-9b6796d9619e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:25.555802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba74db2-d4e6-41af-b630-9b6796d9619e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:25.589925 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a1a534d5-37cd-4f92-aa9b-980d71cc3206 None None] MFA Rules not processed for user `fda45ac8ce02440ba5c6002bac3cd11d`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:25.605044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a1a534d5-37cd-4f92-aa9b-980d71cc3206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:25.610658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 872/1727] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:25.612825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba74db2-d4e6-41af-b630-9b6796d9619e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:25.615641 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0ba74db2-d4e6-41af-b630-9b6796d9619e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:25.634401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:25.645424 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba74db2-d4e6-41af-b630-9b6796d9619e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:25.651136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb02bc3-4ffa-4add-8b3a-20773d883f59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:25.652188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb02bc3-4ffa-4add-8b3a-20773d883f59 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:25.652787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb02bc3-4ffa-4add-8b3a-20773d883f59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:25.653790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb02bc3-4ffa-4add-8b3a-20773d883f59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:25.687292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb02bc3-4ffa-4add-8b3a-20773d883f59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:25.693090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb02bc3-4ffa-4add-8b3a-20773d883f59 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:25.729528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 873/1728] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:25.738107 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb02bc3-4ffa-4add-8b3a-20773d883f59 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:25.761079 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb02bc3-4ffa-4add-8b3a-20773d883f59 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:25.772071 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614ad88-1e3d-4f4f-8448-03052114d0b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:25.772718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614ad88-1e3d-4f4f-8448-03052114d0b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:25.773654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614ad88-1e3d-4f4f-8448-03052114d0b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:25.773943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614ad88-1e3d-4f4f-8448-03052114d0b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:25.797564 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614ad88-1e3d-4f4f-8448-03052114d0b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:25.805703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614ad88-1e3d-4f4f-8448-03052114d0b4 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:25.808119 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 856/1729] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:25.814463 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a534d5-37cd-4f92-aa9b-980d71cc3206 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:25.833764 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 874/1730] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:25.835363 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614ad88-1e3d-4f4f-8448-03052114d0b4 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:25.836748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a534d5-37cd-4f92-aa9b-980d71cc3206 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:25.839986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a614ad88-1e3d-4f4f-8448-03052114d0b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:25.847938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967650f-e143-484c-8a85-77a4e21469ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:25.849176 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967650f-e143-484c-8a85-77a4e21469ff admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:25.849476 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967650f-e143-484c-8a85-77a4e21469ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:25.849754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967650f-e143-484c-8a85-77a4e21469ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:25.850480 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967650f-e143-484c-8a85-77a4e21469ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:25.855583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967650f-e143-484c-8a85-77a4e21469ff admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:25.869087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:25.940468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 857/1731] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:25 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:25.944297 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967650f-e143-484c-8a85-77a4e21469ff admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:26.081912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614ad88-1e3d-4f4f-8448-03052114d0b4 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:26.191532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967650f-e143-484c-8a85-77a4e21469ff admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:26.199349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc54e51-fcbb-444f-91dd-a22d04c91ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:26.216183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc54e51-fcbb-444f-91dd-a22d04c91ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:26.216495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc54e51-fcbb-444f-91dd-a22d04c91ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:26.216769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc54e51-fcbb-444f-91dd-a22d04c91ec2 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:26.224840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc54e51-fcbb-444f-91dd-a22d04c91ec2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:26.226891 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc54e51-fcbb-444f-91dd-a22d04c91ec2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:26.290548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bbaa69-6531-4461-8bd5-1f85c4383cb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:26.291283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbaa69-6531-4461-8bd5-1f85c4383cb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:26.291599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbaa69-6531-4461-8bd5-1f85c4383cb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:26.291931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbaa69-6531-4461-8bd5-1f85c4383cb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:26.309922 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 858/1732] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:25 2022] POST /identity/v3/users => generated 293 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:22:26.313215 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629602c5-7d29-4565-aa58-9ea3fd34c7b5 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:26.313215 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629602c5-7d29-4565-aa58-9ea3fd34c7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:26.313427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629602c5-7d29-4565-aa58-9ea3fd34c7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:26.419931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-629602c5-7d29-4565-aa58-9ea3fd34c7b5 None None] MFA Rules not processed for user `18d196263d2c49f88b5f81acb2586bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:26.422700 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-629602c5-7d29-4565-aa58-9ea3fd34c7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:26.479114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bbaa69-6531-4461-8bd5-1f85c4383cb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:26.480873 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bbaa69-6531-4461-8bd5-1f85c4383cb7 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:26.505595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 875/1733] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:26.506529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bbaa69-6531-4461-8bd5-1f85c4383cb7 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:26.526694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bbaa69-6531-4461-8bd5-1f85c4383cb7 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:26.540255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004a10f-405c-4578-aef7-876edbd94836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:26.541318 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004a10f-405c-4578-aef7-876edbd94836 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:26.541605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004a10f-405c-4578-aef7-876edbd94836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:26.541878 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004a10f-405c-4578-aef7-876edbd94836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:26.542531 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004a10f-405c-4578-aef7-876edbd94836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:26.544212 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004a10f-405c-4578-aef7-876edbd94836 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:26.567699 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 859/1734] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:26.578700 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afdc11-ed12-4f93-a11c-3ce7a9032c9a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:26.578990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afdc11-ed12-4f93-a11c-3ce7a9032c9a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:26.579263 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afdc11-ed12-4f93-a11c-3ce7a9032c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:26.583238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 876/1735] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:26.588376 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004a10f-405c-4578-aef7-876edbd94836 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:26.612165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004a10f-405c-4578-aef7-876edbd94836 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:26.628022 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ab002-4686-4156-95f2-e36b31418319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:26.628677 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ab002-4686-4156-95f2-e36b31418319 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:26.628959 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ab002-4686-4156-95f2-e36b31418319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:26.629238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ab002-4686-4156-95f2-e36b31418319 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:26.629896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ab002-4686-4156-95f2-e36b31418319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:26.631543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ab002-4686-4156-95f2-e36b31418319 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:26.633201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-36afdc11-ed12-4f93-a11c-3ce7a9032c9a None None] MFA Rules not processed for user `18d196263d2c49f88b5f81acb2586bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:26.634550 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-36afdc11-ed12-4f93-a11c-3ce7a9032c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:26.653341 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 860/1736] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:26.659625 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 877/1737] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:26.663247 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afdc11-ed12-4f93-a11c-3ce7a9032c9a None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:26.666542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ab002-4686-4156-95f2-e36b31418319 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:26.677869 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ca6ab002-4686-4156-95f2-e36b31418319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:26.702805 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afdc11-ed12-4f93-a11c-3ce7a9032c9a None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:26.724517 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73175d-d06e-4b5d-8e9b-ae51dc4899b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:26.725439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73175d-d06e-4b5d-8e9b-ae51dc4899b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:26.727037 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73175d-d06e-4b5d-8e9b-ae51dc4899b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:26.728254 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73175d-d06e-4b5d-8e9b-ae51dc4899b4 admin admin] PATH_INFO: `/v3/projects/d86931f134de47d1bb98cddb9cd0fe97/users/dd96383e63ad449dbd5119cbb6bf054a/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:26.799839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73175d-d06e-4b5d-8e9b-ae51dc4899b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d86931f134de47d1bb98cddb9cd0fe97, user_id=dd96383e63ad449dbd5119cbb6bf054a, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:26.802896 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73175d-d06e-4b5d-8e9b-ae51dc4899b4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:26.824917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:26.831563 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 861/1738] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:26 2022] PUT /identity/v3/projects/d86931f134de47d1bb98cddb9cd0fe97/users/dd96383e63ad449dbd5119cbb6bf054a/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:26.843182 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843f1a0-dd20-48c5-8f92-40ab6b32c5c1 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:26.843736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843f1a0-dd20-48c5-8f92-40ab6b32c5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:26.844260 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843f1a0-dd20-48c5-8f92-40ab6b32c5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:26.916783 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7843f1a0-dd20-48c5-8f92-40ab6b32c5c1 None None] MFA Rules not processed for user `dd96383e63ad449dbd5119cbb6bf054a`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:26.923762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7843f1a0-dd20-48c5-8f92-40ab6b32c5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:27.019161 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:27.024699 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ab002-4686-4156-95f2-e36b31418319 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:27.130697 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 862/1739] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:22:27.131712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843f1a0-dd20-48c5-8f92-40ab6b32c5c1 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:27.185677 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bf211-5e69-40ca-a97e-ca12b61f7c66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:27.186315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bf211-5e69-40ca-a97e-ca12b61f7c66 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:27.186601 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bf211-5e69-40ca-a97e-ca12b61f7c66 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:27.186884 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bf211-5e69-40ca-a97e-ca12b61f7c66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:27.246616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843f1a0-dd20-48c5-8f92-40ab6b32c5c1 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:27.252327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becd201-c6e1-4f5e-8989-0236970c51ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:27.253097 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becd201-c6e1-4f5e-8989-0236970c51ab service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:27.253487 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becd201-c6e1-4f5e-8989-0236970c51ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:27.253865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becd201-c6e1-4f5e-8989-0236970c51ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:27.255841 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9becd201-c6e1-4f5e-8989-0236970c51ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:27.334209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bf211-5e69-40ca-a97e-ca12b61f7c66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:27.336981 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bf211-5e69-40ca-a97e-ca12b61f7c66 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:27.361457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 878/1740] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:27.362621 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bf211-5e69-40ca-a97e-ca12b61f7c66 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:27.367584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5b6bf211-5e69-40ca-a97e-ca12b61f7c66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:27.457259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9becd201-c6e1-4f5e-8989-0236970c51ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:27.459701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9becd201-c6e1-4f5e-8989-0236970c51ab service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:27.473607 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 863/1741] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:27.475286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becd201-c6e1-4f5e-8989-0236970c51ab service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:27.504164 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:27.511752 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bf211-5e69-40ca-a97e-ca12b61f7c66 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:27.528876 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475824c-0f1d-4316-970a-1cb42d5fabba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:27.529960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475824c-0f1d-4316-970a-1cb42d5fabba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:27.529960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475824c-0f1d-4316-970a-1cb42d5fabba admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:27.530151 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475824c-0f1d-4316-970a-1cb42d5fabba admin admin] PATH_INFO: `/v3/projects/e8a54c4498624ea4baf34f732a1730d1` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:27.532613 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475824c-0f1d-4316-970a-1cb42d5fabba admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8a54c4498624ea4baf34f732a1730d1)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:27.534550 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475824c-0f1d-4316-970a-1cb42d5fabba admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:27.648095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becd201-c6e1-4f5e-8989-0236970c51ab service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:27.664191 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8475824c-0f1d-4316-970a-1cb42d5fabba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8a54c4498624ea4baf34f732a1730d1', 'request_id': 'req-8475824c-0f1d-4316-970a-1cb42d5fabba'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:27.678850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8475824c-0f1d-4316-970a-1cb42d5fabba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8a54c4498624ea4baf34f732a1730d1', 'request_id': 'req-8475824c-0f1d-4316-970a-1cb42d5fabba'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:27.693386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8475824c-0f1d-4316-970a-1cb42d5fabba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8a54c4498624ea4baf34f732a1730d1', 'request_id': 'req-8475824c-0f1d-4316-970a-1cb42d5fabba'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:27.734167 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8475824c-0f1d-4316-970a-1cb42d5fabba admin admin] The token cache is being invalidate because project e8a54c4498624ea4baf34f732a1730d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:27.734512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8475824c-0f1d-4316-970a-1cb42d5fabba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:27.741405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8475824c-0f1d-4316-970a-1cb42d5fabba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:27.744348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 879/1742] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:27 2022] DELETE /identity/v3/projects/e8a54c4498624ea4baf34f732a1730d1 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:27.745034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475824c-0f1d-4316-970a-1cb42d5fabba admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:27.754816 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8475824c-0f1d-4316-970a-1cb42d5fabba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:27.764105 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:27.850482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd9a2d-5e0b-46a0-8d2b-0c42fab7dcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:27.854782 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd9a2d-5e0b-46a0-8d2b-0c42fab7dcf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:27.860685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd9a2d-5e0b-46a0-8d2b-0c42fab7dcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:27.860982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd9a2d-5e0b-46a0-8d2b-0c42fab7dcf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:27.861697 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd9a2d-5e0b-46a0-8d2b-0c42fab7dcf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:27.893015 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd9a2d-5e0b-46a0-8d2b-0c42fab7dcf6 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:28.001155 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 864/1743] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:27 2022] POST /identity/v3/projects => generated 385 bytes in 524 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:28.002647 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7be7fa-b384-4773-8265-56d14ab6a7ca None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:28.003042 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7be7fa-b384-4773-8265-56d14ab6a7ca None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:28.003429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7be7fa-b384-4773-8265-56d14ab6a7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:28.046727 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ec7be7fa-b384-4773-8265-56d14ab6a7ca None None] MFA Rules not processed for user `1f4cbe342b2c4378b3c0ab2f1cd55260`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:28.049303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ec7be7fa-b384-4773-8265-56d14ab6a7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:28.059336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475824c-0f1d-4316-970a-1cb42d5fabba admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:28.158204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 865/1744] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:28.159868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050f55b-eaa8-4b72-8e2f-b2b7b8889acc None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:28.160715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050f55b-eaa8-4b72-8e2f-b2b7b8889acc None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:28.161237 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050f55b-eaa8-4b72-8e2f-b2b7b8889acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:28.197089 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f050f55b-eaa8-4b72-8e2f-b2b7b8889acc None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:28.209851 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f050f55b-eaa8-4b72-8e2f-b2b7b8889acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:28.220095 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79677ed-d445-414a-b046-97867adbdcb1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:28.221940 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79677ed-d445-414a-b046-97867adbdcb1 service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:28.222227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79677ed-d445-414a-b046-97867adbdcb1 service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:28.222498 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79677ed-d445-414a-b046-97867adbdcb1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:28.228568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a79677ed-d445-414a-b046-97867adbdcb1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:28.260803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79677ed-d445-414a-b046-97867adbdcb1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:28.262377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79677ed-d445-414a-b046-97867adbdcb1 service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:28.301786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 880/1745] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:28.306999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79677ed-d445-414a-b046-97867adbdcb1 service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:28.315174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a79677ed-d445-414a-b046-97867adbdcb1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:28.347019 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 866/1746] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:28.351016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622cab1-277b-4d4b-8376-aef353aa62ac None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:28.351844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622cab1-277b-4d4b-8376-aef353aa62ac None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:28.352854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622cab1-277b-4d4b-8376-aef353aa62ac None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:28.357009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 867/1747] 10.222.0.33 () {54 vars in 912 bytes} [Thu Oct 6 12:22:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:22:28.358587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226b79b-871e-47cd-a092-f95ef294a16d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:28.360879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226b79b-871e-47cd-a092-f95ef294a16d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:28.361273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226b79b-871e-47cd-a092-f95ef294a16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:28.390123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79677ed-d445-414a-b046-97867adbdcb1 service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:28.409147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-1226b79b-871e-47cd-a092-f95ef294a16d None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:28.411756 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ef310-735a-44fd-b8bb-3332c6af5645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:28.412401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ef310-735a-44fd-b8bb-3332c6af5645 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:28.412679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ef310-735a-44fd-b8bb-3332c6af5645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:28.412952 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ef310-735a-44fd-b8bb-3332c6af5645 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:28.413747 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1226b79b-871e-47cd-a092-f95ef294a16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:28.414222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ef310-735a-44fd-b8bb-3332c6af5645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:28.420896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ef310-735a-44fd-b8bb-3332c6af5645 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:28.433418 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 868/1748] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:28.435685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e98d4-c68f-4c69-9cf6-478cc73757ca None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:28.439714 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e98d4-c68f-4c69-9cf6-478cc73757ca None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:28.440042 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e98d4-c68f-4c69-9cf6-478cc73757ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:28.489777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a22e98d4-c68f-4c69-9cf6-478cc73757ca None None] MFA Rules not processed for user `2e71f9c1626f4f58916a5dd5c5987ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:28.495782 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a22e98d4-c68f-4c69-9cf6-478cc73757ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:28.501811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 881/1749] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:28 2022] POST /identity/v3/users => generated 299 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:28.505645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ef310-735a-44fd-b8bb-3332c6af5645 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:28.521604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:28.545722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ef310-735a-44fd-b8bb-3332c6af5645 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:28.573904 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7f2ef-f1d1-4cff-b33f-b636e45694da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:28.574588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7f2ef-f1d1-4cff-b33f-b636e45694da admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:28.578716 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7f2ef-f1d1-4cff-b33f-b636e45694da admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:28.579023 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7f2ef-f1d1-4cff-b33f-b636e45694da admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:28.585161 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7f2ef-f1d1-4cff-b33f-b636e45694da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:28.585766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7f2ef-f1d1-4cff-b33f-b636e45694da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:28.589190 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7f2ef-f1d1-4cff-b33f-b636e45694da admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:28.631097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 882/1750] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:28.631956 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7f2ef-f1d1-4cff-b33f-b636e45694da admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:28.638229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:28.649752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 869/1751] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Oct 6 12:22:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:28.650979 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0753a-9b79-49ef-b83c-d48590a1afc4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:28.651268 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0753a-9b79-49ef-b83c-d48590a1afc4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:28.651598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0753a-9b79-49ef-b83c-d48590a1afc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:28.658989 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7f2ef-f1d1-4cff-b33f-b636e45694da admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:28.669486 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e83e5-ca7e-41fb-bab2-8db8e686cc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:28.673824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e83e5-ca7e-41fb-bab2-8db8e686cc04 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:28.673824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e83e5-ca7e-41fb-bab2-8db8e686cc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:28.674129 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e83e5-ca7e-41fb-bab2-8db8e686cc04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:28.675322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e83e5-ca7e-41fb-bab2-8db8e686cc04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:28.677722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e83e5-ca7e-41fb-bab2-8db8e686cc04 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:28.697691 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-aca0753a-9b79-49ef-b83c-d48590a1afc4 None None] MFA Rules not processed for user `3066c1c9c79942c289b733d886d9fd0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:28.702452 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 883/1752] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:28.703650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e83e5-ca7e-41fb-bab2-8db8e686cc04 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:28.707633 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-aca0753a-9b79-49ef-b83c-d48590a1afc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:28.725926 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e83e5-ca7e-41fb-bab2-8db8e686cc04 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:28.736305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efae333-4bda-47f1-ae8f-ad6defc7198f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:28.736919 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efae333-4bda-47f1-ae8f-ad6defc7198f admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:28.737194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efae333-4bda-47f1-ae8f-ad6defc7198f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:28.737467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efae333-4bda-47f1-ae8f-ad6defc7198f admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:28.738104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efae333-4bda-47f1-ae8f-ad6defc7198f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:28.748435 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efae333-4bda-47f1-ae8f-ad6defc7198f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:28.821053 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 884/1753] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:28 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:28.821053 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efae333-4bda-47f1-ae8f-ad6defc7198f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:28.825672 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7efae333-4bda-47f1-ae8f-ad6defc7198f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:28.838946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 870/1754] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:28.840124 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0753a-9b79-49ef-b83c-d48590a1afc4 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:28.862024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0753a-9b79-49ef-b83c-d48590a1afc4 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:29.018113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efae333-4bda-47f1-ae8f-ad6defc7198f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:29.031057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6feff8-8001-4dff-a6ef-9a6d9d143aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:29.031685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6feff8-8001-4dff-a6ef-9a6d9d143aec admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:29.031975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6feff8-8001-4dff-a6ef-9a6d9d143aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:29.032296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6feff8-8001-4dff-a6ef-9a6d9d143aec admin admin] PATH_INFO: `/v3/projects/3703fd217f8b409eae2436b767ce89cd/users/731a4186e018485cbe860cf0950e2190/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:29.033053 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8abc5-5355-4835-8eac-27250aaa5dcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:29.033840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8abc5-5355-4835-8eac-27250aaa5dcd service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:29.034265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8abc5-5355-4835-8eac-27250aaa5dcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:29.034689 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8abc5-5355-4835-8eac-27250aaa5dcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:29.048938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8abc5-5355-4835-8eac-27250aaa5dcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:29.050897 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8abc5-5355-4835-8eac-27250aaa5dcd service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:29.055217 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6feff8-8001-4dff-a6ef-9a6d9d143aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3703fd217f8b409eae2436b767ce89cd, user_id=731a4186e018485cbe860cf0950e2190, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:29.060455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6feff8-8001-4dff-a6ef-9a6d9d143aec admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:29.071534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 885/1755] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:29.072612 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8abc5-5355-4835-8eac-27250aaa5dcd service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:29.093612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 871/1756] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:28 2022] PUT /identity/v3/projects/3703fd217f8b409eae2436b767ce89cd/users/731a4186e018485cbe860cf0950e2190/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:29.096087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6feff8-8001-4dff-a6ef-9a6d9d143aec admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:29.138329 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8abc5-5355-4835-8eac-27250aaa5dcd service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:29.305561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6feff8-8001-4dff-a6ef-9a6d9d143aec admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:29.316977 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69ba15-7f40-4648-a8f4-34a30c6ea45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:29.318395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69ba15-7f40-4648-a8f4-34a30c6ea45f admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:29.320253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69ba15-7f40-4648-a8f4-34a30c6ea45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:29.320770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69ba15-7f40-4648-a8f4-34a30c6ea45f admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:29.325565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69ba15-7f40-4648-a8f4-34a30c6ea45f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:29.327906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268531ed-b054-42a7-81e5-6a12cc7a24b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:29.328814 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268531ed-b054-42a7-81e5-6a12cc7a24b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:29.329312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268531ed-b054-42a7-81e5-6a12cc7a24b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:29.330639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268531ed-b054-42a7-81e5-6a12cc7a24b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:29.331060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69ba15-7f40-4648-a8f4-34a30c6ea45f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:29.331956 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268531ed-b054-42a7-81e5-6a12cc7a24b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:29.336518 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268531ed-b054-42a7-81e5-6a12cc7a24b0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:29.365168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 872/1757] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:29.372204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268531ed-b054-42a7-81e5-6a12cc7a24b0 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:29.388021 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 886/1758] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:29 2022] POST /identity/v3/users => generated 294 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:29.389468 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69ba15-7f40-4648-a8f4-34a30c6ea45f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:29.401109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268531ed-b054-42a7-81e5-6a12cc7a24b0 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:29.409391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79d560-fe10-4ca7-9684-bbe322a71d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:29.411173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79d560-fe10-4ca7-9684-bbe322a71d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:29.411844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79d560-fe10-4ca7-9684-bbe322a71d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:29.412696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79d560-fe10-4ca7-9684-bbe322a71d33 admin admin] PATH_INFO: `/v3/projects/3703fd217f8b409eae2436b767ce89cd/users/731a4186e018485cbe860cf0950e2190/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:29.416246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79d560-fe10-4ca7-9684-bbe322a71d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3703fd217f8b409eae2436b767ce89cd, user_id=731a4186e018485cbe860cf0950e2190, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:29.417818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69ba15-7f40-4648-a8f4-34a30c6ea45f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:29.421349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79d560-fe10-4ca7-9684-bbe322a71d33 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:29.435812 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee895eb0-4481-4cce-8636-b175eb9381fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:29.439117 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee895eb0-4481-4cce-8636-b175eb9381fb admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:29.441113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee895eb0-4481-4cce-8636-b175eb9381fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:29.441402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee895eb0-4481-4cce-8636-b175eb9381fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:29.444857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 873/1759] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:29 2022] PUT /identity/v3/projects/3703fd217f8b409eae2436b767ce89cd/users/731a4186e018485cbe860cf0950e2190/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:29.446588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee895eb0-4481-4cce-8636-b175eb9381fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:29.453262 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee895eb0-4481-4cce-8636-b175eb9381fb admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:29.457266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d1e2c-e8f7-4092-be26-2cdb5e01aea0 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:29.462502 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d1e2c-e8f7-4092-be26-2cdb5e01aea0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:29.462502 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d1e2c-e8f7-4092-be26-2cdb5e01aea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:29.495741 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 887/1760] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:29.498151 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee895eb0-4481-4cce-8636-b175eb9381fb admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:29.510525 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:29.532272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee895eb0-4481-4cce-8636-b175eb9381fb admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:29.534733 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-b30d1e2c-e8f7-4092-be26-2cdb5e01aea0 None None] MFA Rules not processed for user `731a4186e018485cbe860cf0950e2190`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:29.535775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b30d1e2c-e8f7-4092-be26-2cdb5e01aea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:29.619745 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:29.655149 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7d742-61c0-467c-a49c-efffa00e9c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:29.655866 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7d742-61c0-467c-a49c-efffa00e9c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:29.656146 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7d742-61c0-467c-a49c-efffa00e9c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:29.656419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7d742-61c0-467c-a49c-efffa00e9c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:29.657069 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7d742-61c0-467c-a49c-efffa00e9c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:29.659937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7d742-61c0-467c-a49c-efffa00e9c10 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:29.679554 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 888/1761] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:29.683481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7d742-61c0-467c-a49c-efffa00e9c10 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:29.710306 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7d742-61c0-467c-a49c-efffa00e9c10 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:29.724131 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 874/1762] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:29.740555 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d1e2c-e8f7-4092-be26-2cdb5e01aea0 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:29.743026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d5410f-d1b5-4ddc-91d8-e08bb44f0982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:29.743746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5410f-d1b5-4ddc-91d8-e08bb44f0982 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:29.744033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b30d1e2c-e8f7-4092-be26-2cdb5e01aea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:29.744152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5410f-d1b5-4ddc-91d8-e08bb44f0982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:29.744298 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5410f-d1b5-4ddc-91d8-e08bb44f0982 admin admin] PATH_INFO: `/v3/projects/233246d10f58498b9bdcc583708fb718/users/61d2bce3768a49f788b0bdaef5ec3ab8/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:29.798173 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5410f-d1b5-4ddc-91d8-e08bb44f0982 admin admin] RBAC: Authorizing `identity:create_grant(project_id=233246d10f58498b9bdcc583708fb718, user_id=61d2bce3768a49f788b0bdaef5ec3ab8, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:29.803903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5410f-d1b5-4ddc-91d8-e08bb44f0982 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:29.834549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 889/1763] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:29 2022] PUT /identity/v3/projects/233246d10f58498b9bdcc583708fb718/users/61d2bce3768a49f788b0bdaef5ec3ab8/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:29.846935 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e293b-8c10-4070-b1eb-1d5fb3999150 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:29.847106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e293b-8c10-4070-b1eb-1d5fb3999150 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:29.847423 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e293b-8c10-4070-b1eb-1d5fb3999150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:29.933758 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-261e293b-8c10-4070-b1eb-1d5fb3999150 None None] MFA Rules not processed for user `61d2bce3768a49f788b0bdaef5ec3ab8`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:29.939010 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-261e293b-8c10-4070-b1eb-1d5fb3999150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:29.963091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d1e2c-e8f7-4092-be26-2cdb5e01aea0 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:30.137303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a3de4-f4aa-4407-91d2-dda6dedeea29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:30.138161 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a3de4-f4aa-4407-91d2-dda6dedeea29 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:30.138413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a3de4-f4aa-4407-91d2-dda6dedeea29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:30.138823 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a3de4-f4aa-4407-91d2-dda6dedeea29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:30.220484 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 890/1764] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:22:30.225730 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e293b-8c10-4070-b1eb-1d5fb3999150 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:30.388644 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a3de4-f4aa-4407-91d2-dda6dedeea29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:30.391944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a3de4-f4aa-4407-91d2-dda6dedeea29 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:30.427536 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 875/1765] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 686 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:30.470589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e293b-8c10-4070-b1eb-1d5fb3999150 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:30.490174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a3de4-f4aa-4407-91d2-dda6dedeea29 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:30.495408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:30.500651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdbb80-640f-4955-9732-7969d928b93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:30.501568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdbb80-640f-4955-9732-7969d928b93b admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:30.501857 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdbb80-640f-4955-9732-7969d928b93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:30.502139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdbb80-640f-4955-9732-7969d928b93b admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:30.502770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fdbb80-640f-4955-9732-7969d928b93b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:30.504746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fdbb80-640f-4955-9732-7969d928b93b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:30.518523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a3de4-f4aa-4407-91d2-dda6dedeea29 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:30.526441 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bfbbe-e20f-456f-b704-e7f095530ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:30.527098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bfbbe-e20f-456f-b704-e7f095530ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:30.527381 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bfbbe-e20f-456f-b704-e7f095530ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:30.527684 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bfbbe-e20f-456f-b704-e7f095530ceb admin admin] PATH_INFO: `/v3/users/dd96383e63ad449dbd5119cbb6bf054a` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:30.533482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0bfbbe-e20f-456f-b704-e7f095530ceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd96383e63ad449dbd5119cbb6bf054a)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:30.536579 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0bfbbe-e20f-456f-b704-e7f095530ceb admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:30.616734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 891/1766] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:30 2022] POST /identity/v3/projects => generated 387 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:30.617419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdbb80-640f-4955-9732-7969d928b93b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:30.708650 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fb0bfbbe-e20f-456f-b704-e7f095530ceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd96383e63ad449dbd5119cbb6bf054a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:30.726679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fb0bfbbe-e20f-456f-b704-e7f095530ceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd96383e63ad449dbd5119cbb6bf054a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:30.727969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fb0bfbbe-e20f-456f-b704-e7f095530ceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd96383e63ad449dbd5119cbb6bf054a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:30.730806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:30.731909 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fb0bfbbe-e20f-456f-b704-e7f095530ceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd96383e63ad449dbd5119cbb6bf054a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:30.752972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fb0bfbbe-e20f-456f-b704-e7f095530ceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd96383e63ad449dbd5119cbb6bf054a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:30.796889 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 876/1767] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:30 2022] DELETE /identity/v3/users/dd96383e63ad449dbd5119cbb6bf054a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:30.799069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babb5f0-101a-43b3-a26d-518c591d6733 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:30.799354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babb5f0-101a-43b3-a26d-518c591d6733 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:30.800519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babb5f0-101a-43b3-a26d-518c591d6733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:30.816553 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdbb80-640f-4955-9732-7969d928b93b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:30.847273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-1babb5f0-101a-43b3-a26d-518c591d6733 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:30.849402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1babb5f0-101a-43b3-a26d-518c591d6733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:31.019043 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff479bc-eb33-4731-992d-e4f16b859404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:31.020156 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff479bc-eb33-4731-992d-e4f16b859404 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:31.021923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff479bc-eb33-4731-992d-e4f16b859404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:31.022447 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff479bc-eb33-4731-992d-e4f16b859404 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:31.023410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff479bc-eb33-4731-992d-e4f16b859404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:31.026687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff479bc-eb33-4731-992d-e4f16b859404 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:31.063571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 877/1768] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:31.070492 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babb5f0-101a-43b3-a26d-518c591d6733 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:31.072104 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1babb5f0-101a-43b3-a26d-518c591d6733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:31.078836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 892/1769] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:30 2022] POST /identity/v3/users => generated 300 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:31.082884 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff479bc-eb33-4731-992d-e4f16b859404 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:31.087088 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5ff479bc-eb33-4731-992d-e4f16b859404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:31.117432 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff479bc-eb33-4731-992d-e4f16b859404 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:31.126406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60544aa-280b-4c37-8dcc-fc28b38701ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:31.127082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60544aa-280b-4c37-8dcc-fc28b38701ce admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:31.127370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60544aa-280b-4c37-8dcc-fc28b38701ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:31.127719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60544aa-280b-4c37-8dcc-fc28b38701ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:31.129577 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60544aa-280b-4c37-8dcc-fc28b38701ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:31.131323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60544aa-280b-4c37-8dcc-fc28b38701ce admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:31.156001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 893/1770] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:31.159935 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60544aa-280b-4c37-8dcc-fc28b38701ce admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:31.171870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60544aa-280b-4c37-8dcc-fc28b38701ce admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:31.178319 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2db30-1ab7-46a0-99ac-a70e9fac608f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:31.179161 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2db30-1ab7-46a0-99ac-a70e9fac608f admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:31.179714 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2db30-1ab7-46a0-99ac-a70e9fac608f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:31.180065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2db30-1ab7-46a0-99ac-a70e9fac608f admin admin] PATH_INFO: `/v3/projects/4bbba53d79b14d64a04ab0be9562bd92/users/efd80044b6704e059f037a8ed36ae38e/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:31.224275 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babb5f0-101a-43b3-a26d-518c591d6733 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:31.225592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2db30-1ab7-46a0-99ac-a70e9fac608f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bbba53d79b14d64a04ab0be9562bd92, user_id=efd80044b6704e059f037a8ed36ae38e, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:31.229814 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2db30-1ab7-46a0-99ac-a70e9fac608f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:31.240892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcece81-8de3-42fb-813d-3f626902ccdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:31.242085 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcece81-8de3-42fb-813d-3f626902ccdd service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:31.242575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcece81-8de3-42fb-813d-3f626902ccdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:31.243222 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcece81-8de3-42fb-813d-3f626902ccdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:31.260081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcece81-8de3-42fb-813d-3f626902ccdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:31.261698 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcece81-8de3-42fb-813d-3f626902ccdd service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:31.278966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 878/1771] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:22:31.305474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 894/1772] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:31 2022] PUT /identity/v3/projects/4bbba53d79b14d64a04ab0be9562bd92/users/efd80044b6704e059f037a8ed36ae38e/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:31.310338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcece81-8de3-42fb-813d-3f626902ccdd service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:31.322244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcece81-8de3-42fb-813d-3f626902ccdd service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:31.346523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:31.441234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5971d-a6e5-4387-88ba-1089c4429f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:31.442303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5971d-a6e5-4387-88ba-1089c4429f2e admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:31.442808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5971d-a6e5-4387-88ba-1089c4429f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:31.443928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5971d-a6e5-4387-88ba-1089c4429f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:31.445563 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5971d-a6e5-4387-88ba-1089c4429f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:31.449619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5971d-a6e5-4387-88ba-1089c4429f2e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:31.495842 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 879/1773] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:31.499770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2db30-1ab7-46a0-99ac-a70e9fac608f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:31.517954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2db30-1ab7-46a0-99ac-a70e9fac608f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:31.529040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295243e-a009-4e5e-943a-05c6d6c835fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:31.529734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295243e-a009-4e5e-943a-05c6d6c835fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:31.530015 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295243e-a009-4e5e-943a-05c6d6c835fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:31.530307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295243e-a009-4e5e-943a-05c6d6c835fa admin admin] PATH_INFO: `/v3/projects/4bbba53d79b14d64a04ab0be9562bd92/users/efd80044b6704e059f037a8ed36ae38e/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:31.536794 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295243e-a009-4e5e-943a-05c6d6c835fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bbba53d79b14d64a04ab0be9562bd92, user_id=efd80044b6704e059f037a8ed36ae38e, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:31.542124 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295243e-a009-4e5e-943a-05c6d6c835fa admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:31.574210 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 895/1774] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:31 2022] PUT /identity/v3/projects/4bbba53d79b14d64a04ab0be9562bd92/users/efd80044b6704e059f037a8ed36ae38e/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:31.586970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c42e07-9dfe-4f36-8922-fd1470f50ab0 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:31.587812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c42e07-9dfe-4f36-8922-fd1470f50ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:31.590129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c42e07-9dfe-4f36-8922-fd1470f50ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:31.686767 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-b4c42e07-9dfe-4f36-8922-fd1470f50ab0 None None] MFA Rules not processed for user `efd80044b6704e059f037a8ed36ae38e`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:31.688147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b4c42e07-9dfe-4f36-8922-fd1470f50ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:31.882577 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 880/1775] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:31.897399 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295243e-a009-4e5e-943a-05c6d6c835fa admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:31.900022 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3295243e-a009-4e5e-943a-05c6d6c835fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:31.993396 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c42e07-9dfe-4f36-8922-fd1470f50ab0 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:31.995215 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b4c42e07-9dfe-4f36-8922-fd1470f50ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:32.122464 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295243e-a009-4e5e-943a-05c6d6c835fa admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:32.127384 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389951d-6a88-4d85-bfa1-f9d220da43a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:32.128223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389951d-6a88-4d85-bfa1-f9d220da43a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:32.128599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389951d-6a88-4d85-bfa1-f9d220da43a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:32.129439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389951d-6a88-4d85-bfa1-f9d220da43a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:32.142403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389951d-6a88-4d85-bfa1-f9d220da43a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:32.144939 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389951d-6a88-4d85-bfa1-f9d220da43a9 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:32.158281 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c42e07-9dfe-4f36-8922-fd1470f50ab0 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:32.160598 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 896/1776] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:32.185233 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff78800-751f-46ac-b721-ce0d9348b4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:32.186047 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff78800-751f-46ac-b721-ce0d9348b4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:32.186332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff78800-751f-46ac-b721-ce0d9348b4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:32.186609 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff78800-751f-46ac-b721-ce0d9348b4ef admin admin] PATH_INFO: `/v3/projects/d86931f134de47d1bb98cddb9cd0fe97` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:32.188744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff78800-751f-46ac-b721-ce0d9348b4ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=d86931f134de47d1bb98cddb9cd0fe97)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:32.189906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff78800-751f-46ac-b721-ce0d9348b4ef admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:32.310136 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:32.318165 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6ff78800-751f-46ac-b721-ce0d9348b4ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd86931f134de47d1bb98cddb9cd0fe97', 'request_id': 'req-6ff78800-751f-46ac-b721-ce0d9348b4ef'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:32.333912 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6ff78800-751f-46ac-b721-ce0d9348b4ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd86931f134de47d1bb98cddb9cd0fe97', 'request_id': 'req-6ff78800-751f-46ac-b721-ce0d9348b4ef'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:32.343018 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6ff78800-751f-46ac-b721-ce0d9348b4ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd86931f134de47d1bb98cddb9cd0fe97', 'request_id': 'req-6ff78800-751f-46ac-b721-ce0d9348b4ef'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:32.362318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6ff78800-751f-46ac-b721-ce0d9348b4ef admin admin] The token cache is being invalidate because project d86931f134de47d1bb98cddb9cd0fe97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:32.362699 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6ff78800-751f-46ac-b721-ce0d9348b4ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:32.363667 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6ff78800-751f-46ac-b721-ce0d9348b4ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:32.368319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 881/1777] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:31 2022] DELETE /identity/v3/projects/d86931f134de47d1bb98cddb9cd0fe97 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:32.371396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389951d-6a88-4d85-bfa1-f9d220da43a9 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:32.394250 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff78800-751f-46ac-b721-ce0d9348b4ef admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:32.396907 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389951d-6a88-4d85-bfa1-f9d220da43a9 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:32.397183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6ff78800-751f-46ac-b721-ce0d9348b4ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:32.435881 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:32.521938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed5ec8-647e-472c-bd14-e94003a2caf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:32.523068 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed5ec8-647e-472c-bd14-e94003a2caf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:32.523364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed5ec8-647e-472c-bd14-e94003a2caf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:32.525360 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed5ec8-647e-472c-bd14-e94003a2caf4 admin admin] PATH_INFO: `/v3/users/731a4186e018485cbe860cf0950e2190` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:32.532424 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed5ec8-647e-472c-bd14-e94003a2caf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=731a4186e018485cbe860cf0950e2190)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:32.534726 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed5ec8-647e-472c-bd14-e94003a2caf4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:32.610484 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff78800-751f-46ac-b721-ce0d9348b4ef admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:32.625024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b34023-8871-48ae-92e3-4a1ae133f8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:32.625682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b34023-8871-48ae-92e3-4a1ae133f8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:32.625966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b34023-8871-48ae-92e3-4a1ae133f8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:32.626239 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b34023-8871-48ae-92e3-4a1ae133f8e1 admin admin] PATH_INFO: `/v3/users/61d2bce3768a49f788b0bdaef5ec3ab8` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:32.628683 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9aed5ec8-647e-472c-bd14-e94003a2caf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '731a4186e018485cbe860cf0950e2190'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:32.634793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b34023-8871-48ae-92e3-4a1ae133f8e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61d2bce3768a49f788b0bdaef5ec3ab8)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:32.636775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b34023-8871-48ae-92e3-4a1ae133f8e1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:32.638907 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9aed5ec8-647e-472c-bd14-e94003a2caf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '731a4186e018485cbe860cf0950e2190'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:32.639850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9aed5ec8-647e-472c-bd14-e94003a2caf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '731a4186e018485cbe860cf0950e2190'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:32.640343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9aed5ec8-647e-472c-bd14-e94003a2caf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '731a4186e018485cbe860cf0950e2190'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:32.654501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9aed5ec8-647e-472c-bd14-e94003a2caf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '731a4186e018485cbe860cf0950e2190'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:32.698952 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 897/1778] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:32 2022] DELETE /identity/v3/users/731a4186e018485cbe860cf0950e2190 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:32.733315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-05b34023-8871-48ae-92e3-4a1ae133f8e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61d2bce3768a49f788b0bdaef5ec3ab8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:32.748082 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-05b34023-8871-48ae-92e3-4a1ae133f8e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61d2bce3768a49f788b0bdaef5ec3ab8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:32.749923 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-05b34023-8871-48ae-92e3-4a1ae133f8e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61d2bce3768a49f788b0bdaef5ec3ab8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:32.751970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-05b34023-8871-48ae-92e3-4a1ae133f8e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61d2bce3768a49f788b0bdaef5ec3ab8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:32.769210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-05b34023-8871-48ae-92e3-4a1ae133f8e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61d2bce3768a49f788b0bdaef5ec3ab8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:32.814481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 882/1779] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:32 2022] DELETE /identity/v3/users/61d2bce3768a49f788b0bdaef5ec3ab8 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:32.818063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4248d8a-30f6-4c80-9f0b-ebcc1eff4382 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:32.818355 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4248d8a-30f6-4c80-9f0b-ebcc1eff4382 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:32.818633 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4248d8a-30f6-4c80-9f0b-ebcc1eff4382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:32.850397 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d4248d8a-30f6-4c80-9f0b-ebcc1eff4382 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:32.852247 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d4248d8a-30f6-4c80-9f0b-ebcc1eff4382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:32.985075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 898/1780] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:32.996829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b34023-8871-48ae-92e3-4a1ae133f8e1 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:32.999628 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-05b34023-8871-48ae-92e3-4a1ae133f8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:33.099850 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:33.106920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b34023-8871-48ae-92e3-4a1ae133f8e1 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:33.114056 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc999ab3-1414-4811-855a-cc02eca38ceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:33.114896 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc999ab3-1414-4811-855a-cc02eca38ceb service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:33.115364 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc999ab3-1414-4811-855a-cc02eca38ceb service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:33.116990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc999ab3-1414-4811-855a-cc02eca38ceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:33.131619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc999ab3-1414-4811-855a-cc02eca38ceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:33.134188 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc999ab3-1414-4811-855a-cc02eca38ceb service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:33.152465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 883/1781] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:33.472299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4248d8a-30f6-4c80-9f0b-ebcc1eff4382 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:33.474426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d4248d8a-30f6-4c80-9f0b-ebcc1eff4382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:33.492843 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4248d8a-30f6-4c80-9f0b-ebcc1eff4382 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:33.508151 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496a44f-05ac-4890-9df7-7a7738d40e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:33.509009 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496a44f-05ac-4890-9df7-7a7738d40e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:33.509401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496a44f-05ac-4890-9df7-7a7738d40e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:33.509898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496a44f-05ac-4890-9df7-7a7738d40e5d admin admin] PATH_INFO: `/v3/projects/3703fd217f8b409eae2436b767ce89cd` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:33.514338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496a44f-05ac-4890-9df7-7a7738d40e5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3703fd217f8b409eae2436b767ce89cd)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:33.530281 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496a44f-05ac-4890-9df7-7a7738d40e5d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:33.659894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:33.685091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-f496a44f-05ac-4890-9df7-7a7738d40e5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3703fd217f8b409eae2436b767ce89cd', 'request_id': 'req-f496a44f-05ac-4890-9df7-7a7738d40e5d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:33.698855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-f496a44f-05ac-4890-9df7-7a7738d40e5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3703fd217f8b409eae2436b767ce89cd', 'request_id': 'req-f496a44f-05ac-4890-9df7-7a7738d40e5d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:33.710849 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-f496a44f-05ac-4890-9df7-7a7738d40e5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3703fd217f8b409eae2436b767ce89cd', 'request_id': 'req-f496a44f-05ac-4890-9df7-7a7738d40e5d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:33.741012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-f496a44f-05ac-4890-9df7-7a7738d40e5d admin admin] The token cache is being invalidate because project 3703fd217f8b409eae2436b767ce89cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:33.741312 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-f496a44f-05ac-4890-9df7-7a7738d40e5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:33.742135 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-f496a44f-05ac-4890-9df7-7a7738d40e5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:33.746653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 899/1782] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:33 2022] DELETE /identity/v3/projects/3703fd217f8b409eae2436b767ce89cd => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:33.751745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73487942-3f2d-4c5a-8e72-ea35a0445392 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:33.752261 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73487942-3f2d-4c5a-8e72-ea35a0445392 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:33.752355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73487942-3f2d-4c5a-8e72-ea35a0445392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:33.797473 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-73487942-3f2d-4c5a-8e72-ea35a0445392 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:33.799279 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-73487942-3f2d-4c5a-8e72-ea35a0445392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:33.890575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496a44f-05ac-4890-9df7-7a7738d40e5d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:33.892511 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f496a44f-05ac-4890-9df7-7a7738d40e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:33.948795 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 884/1783] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:33.964114 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79179d73-3788-4391-863c-89d16108cb32 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:33.964318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79179d73-3788-4391-863c-89d16108cb32 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:33.964814 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79179d73-3788-4391-863c-89d16108cb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:34.006795 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-79179d73-3788-4391-863c-89d16108cb32 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:34.009103 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-79179d73-3788-4391-863c-89d16108cb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:34.027123 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 885/1784] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:34.033844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79179d73-3788-4391-863c-89d16108cb32 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:34.057925 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79179d73-3788-4391-863c-89d16108cb32 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:34.063307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496a44f-05ac-4890-9df7-7a7738d40e5d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:34.069400 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be315b-82c4-40e0-8ea5-2c4dc12d883d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:34.073184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be315b-82c4-40e0-8ea5-2c4dc12d883d admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:34.073472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be315b-82c4-40e0-8ea5-2c4dc12d883d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:34.073748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be315b-82c4-40e0-8ea5-2c4dc12d883d admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:34.074424 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be315b-82c4-40e0-8ea5-2c4dc12d883d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:34.075952 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be315b-82c4-40e0-8ea5-2c4dc12d883d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:34.078122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be315b-82c4-40e0-8ea5-2c4dc12d883d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:34.080542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:34.081796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:34.082176 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:34.083841 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4 admin admin] PATH_INFO: `/v3/projects/233246d10f58498b9bdcc583708fb718` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:34.085408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=233246d10f58498b9bdcc583708fb718)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:34.088291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:34.098952 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 886/1785] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:34.103828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be315b-82c4-40e0-8ea5-2c4dc12d883d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:34.122318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be315b-82c4-40e0-8ea5-2c4dc12d883d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:34.128265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa29519-59d8-48c9-b0f5-cc725065edd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:34.129044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa29519-59d8-48c9-b0f5-cc725065edd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:34.129333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa29519-59d8-48c9-b0f5-cc725065edd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:34.129605 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa29519-59d8-48c9-b0f5-cc725065edd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:34.130298 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa29519-59d8-48c9-b0f5-cc725065edd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:34.132450 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa29519-59d8-48c9-b0f5-cc725065edd8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:34.194178 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 887/1786] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:34 2022] POST /identity/v3/projects => generated 389 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:34.200118 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa29519-59d8-48c9-b0f5-cc725065edd8 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:34.254759 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '233246d10f58498b9bdcc583708fb718', 'request_id': 'req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:34.278450 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '233246d10f58498b9bdcc583708fb718', 'request_id': 'req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:34.313140 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '233246d10f58498b9bdcc583708fb718', 'request_id': 'req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:34.330847 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4 admin admin] The token cache is being invalidate because project 233246d10f58498b9bdcc583708fb718 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:34.331219 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:34.332218 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:34.333954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 900/1787] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:33 2022] DELETE /identity/v3/projects/233246d10f58498b9bdcc583708fb718 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:34.338653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:34.341206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:34.345326 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa29519-59d8-48c9-b0f5-cc725065edd8 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:34.360133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87ffe8-c39f-4926-8c78-5a105f52af76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:34.360767 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87ffe8-c39f-4926-8c78-5a105f52af76 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:34.361052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87ffe8-c39f-4926-8c78-5a105f52af76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:34.361327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87ffe8-c39f-4926-8c78-5a105f52af76 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:34.361968 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87ffe8-c39f-4926-8c78-5a105f52af76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:34.364606 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87ffe8-c39f-4926-8c78-5a105f52af76 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:34.427630 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 888/1788] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:34 2022] POST /identity/v3/users => generated 295 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:34.437534 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87ffe8-c39f-4926-8c78-5a105f52af76 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:34.455347 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7f87ffe8-c39f-4926-8c78-5a105f52af76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:34.502857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87ffe8-c39f-4926-8c78-5a105f52af76 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:34.540406 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd69437-90c5-48fc-8a56-0dbfa0c859a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:34.543977 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd69437-90c5-48fc-8a56-0dbfa0c859a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:34.544341 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd69437-90c5-48fc-8a56-0dbfa0c859a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:34.544697 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd69437-90c5-48fc-8a56-0dbfa0c859a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:34.551815 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd69437-90c5-48fc-8a56-0dbfa0c859a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:34.556457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:34.563543 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd69437-90c5-48fc-8a56-0dbfa0c859a1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:34.591650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7dd3f-2a85-4799-9dfd-05a89c0677f4 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:34.612141 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe7c9c-9238-4928-8ead-34c28b0e869f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:34.612876 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe7c9c-9238-4928-8ead-34c28b0e869f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:34.613234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe7c9c-9238-4928-8ead-34c28b0e869f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:34.613679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe7c9c-9238-4928-8ead-34c28b0e869f admin admin] PATH_INFO: `/v3/users/efd80044b6704e059f037a8ed36ae38e` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:34.614905 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe7c9c-9238-4928-8ead-34c28b0e869f admin admin] RBAC: Authorizing `identity:delete_user(user_id=efd80044b6704e059f037a8ed36ae38e)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:34.635200 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 889/1789] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:34.636004 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe7c9c-9238-4928-8ead-34c28b0e869f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:34.638228 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd69437-90c5-48fc-8a56-0dbfa0c859a1 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:34.670852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd69437-90c5-48fc-8a56-0dbfa0c859a1 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:34.688784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf83584-b0ae-41d8-95fa-2cbfed7dba2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:34.689415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf83584-b0ae-41d8-95fa-2cbfed7dba2e admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:34.689694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf83584-b0ae-41d8-95fa-2cbfed7dba2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:34.689973 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf83584-b0ae-41d8-95fa-2cbfed7dba2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:34.690633 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf83584-b0ae-41d8-95fa-2cbfed7dba2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:34.692270 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf83584-b0ae-41d8-95fa-2cbfed7dba2e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:34.724656 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 890/1790] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:34.732972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf83584-b0ae-41d8-95fa-2cbfed7dba2e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:34.736901 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6ebe7c9c-9238-4928-8ead-34c28b0e869f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efd80044b6704e059f037a8ed36ae38e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:34.748561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6ebe7c9c-9238-4928-8ead-34c28b0e869f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efd80044b6704e059f037a8ed36ae38e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:34.749556 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6ebe7c9c-9238-4928-8ead-34c28b0e869f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efd80044b6704e059f037a8ed36ae38e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:34.750858 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6ebe7c9c-9238-4928-8ead-34c28b0e869f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efd80044b6704e059f037a8ed36ae38e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:34.757034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf83584-b0ae-41d8-95fa-2cbfed7dba2e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:34.765217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6ebe7c9c-9238-4928-8ead-34c28b0e869f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efd80044b6704e059f037a8ed36ae38e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:34.766426 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594d580-1697-464d-8717-86cf8e31bf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:34.768100 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594d580-1697-464d-8717-86cf8e31bf59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:34.769494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594d580-1697-464d-8717-86cf8e31bf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:34.769877 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594d580-1697-464d-8717-86cf8e31bf59 admin admin] PATH_INFO: `/v3/projects/4305357752b24eb5a88a1ac84f506dd7/users/5c87a8fd64e84dce9a3198bbaf1e5984/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:34.815567 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 901/1791] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:34 2022] DELETE /identity/v3/users/efd80044b6704e059f037a8ed36ae38e => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:34.841670 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594d580-1697-464d-8717-86cf8e31bf59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4305357752b24eb5a88a1ac84f506dd7, user_id=5c87a8fd64e84dce9a3198bbaf1e5984, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:34.846437 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594d580-1697-464d-8717-86cf8e31bf59 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:34.853025 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:34.897808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 891/1792] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:34 2022] PUT /identity/v3/projects/4305357752b24eb5a88a1ac84f506dd7/users/5c87a8fd64e84dce9a3198bbaf1e5984/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:34.901417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f17e9b-d149-441a-a591-d0e4e4917287 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:34.901863 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f17e9b-d149-441a-a591-d0e4e4917287 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:34.902834 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f17e9b-d149-441a-a591-d0e4e4917287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:34.946774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-58f17e9b-d149-441a-a591-d0e4e4917287 None None] MFA Rules not processed for user `5c87a8fd64e84dce9a3198bbaf1e5984`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:34.948365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-58f17e9b-d149-441a-a591-d0e4e4917287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:35.133169 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5147c06-737a-410e-b928-2fe4ae7d4515 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:35.133669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5147c06-737a-410e-b928-2fe4ae7d4515 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:35.134097 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5147c06-737a-410e-b928-2fe4ae7d4515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:35.142089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 902/1793] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:35.147045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f17e9b-d149-441a-a591-d0e4e4917287 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:35.149157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-58f17e9b-d149-441a-a591-d0e4e4917287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:35.286334 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f5147c06-737a-410e-b928-2fe4ae7d4515 None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:35.288747 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f5147c06-737a-410e-b928-2fe4ae7d4515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:35.356222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:35.374214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f17e9b-d149-441a-a591-d0e4e4917287 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:35.389537 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84505b-8fb4-4a67-9592-1c7a8c09863a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:35.390483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84505b-8fb4-4a67-9592-1c7a8c09863a admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:35.390947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84505b-8fb4-4a67-9592-1c7a8c09863a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:35.391400 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84505b-8fb4-4a67-9592-1c7a8c09863a admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:35.392272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84505b-8fb4-4a67-9592-1c7a8c09863a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:35.396342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84505b-8fb4-4a67-9592-1c7a8c09863a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:35.491382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 903/1794] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:35 2022] POST /identity/v3/projects => generated 389 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:35.495485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84505b-8fb4-4a67-9592-1c7a8c09863a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:35.617624 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84505b-8fb4-4a67-9592-1c7a8c09863a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:35.633587 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abfa59b-21d9-45ea-b074-bf69e75f5cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:35.634250 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfa59b-21d9-45ea-b074-bf69e75f5cae admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:35.634532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfa59b-21d9-45ea-b074-bf69e75f5cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:35.634807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfa59b-21d9-45ea-b074-bf69e75f5cae admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:35.638326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abfa59b-21d9-45ea-b074-bf69e75f5cae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:35.641514 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abfa59b-21d9-45ea-b074-bf69e75f5cae admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:35.707049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 904/1795] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:35 2022] POST /identity/v3/users => generated 301 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:22:35.713311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abfa59b-21d9-45ea-b074-bf69e75f5cae admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:35.738513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 892/1796] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Oct 6 12:22:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 606 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:22:35.740370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abfa59b-21d9-45ea-b074-bf69e75f5cae admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:35.748460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb4798-f7e5-475d-a4b3-4f929d22f364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:35.749458 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb4798-f7e5-475d-a4b3-4f929d22f364 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:35.749834 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb4798-f7e5-475d-a4b3-4f929d22f364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:35.750286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb4798-f7e5-475d-a4b3-4f929d22f364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:35.750973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb4798-f7e5-475d-a4b3-4f929d22f364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:35.753153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb4798-f7e5-475d-a4b3-4f929d22f364 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:35.766019 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5147c06-737a-410e-b928-2fe4ae7d4515 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:35.770226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f5147c06-737a-410e-b928-2fe4ae7d4515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:35.771129 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 905/1797] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:35.773411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb4798-f7e5-475d-a4b3-4f929d22f364 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:35.801652 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb4798-f7e5-475d-a4b3-4f929d22f364 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:35.808699 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f093a53-7d3c-45f9-a89f-6f0916992964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:35.814546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f093a53-7d3c-45f9-a89f-6f0916992964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:35.820759 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f093a53-7d3c-45f9-a89f-6f0916992964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:35.821066 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f093a53-7d3c-45f9-a89f-6f0916992964 admin admin] PATH_INFO: `/v3/projects/db879a8f7bcf4e838da6d2fa19a8483c/users/a544142f73854f2c87c8c26841456074/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:35.846747 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f093a53-7d3c-45f9-a89f-6f0916992964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db879a8f7bcf4e838da6d2fa19a8483c, user_id=a544142f73854f2c87c8c26841456074, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:35.849656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f093a53-7d3c-45f9-a89f-6f0916992964 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:35.854611 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5147c06-737a-410e-b928-2fe4ae7d4515 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:35.869649 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 906/1798] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:35 2022] PUT /identity/v3/projects/db879a8f7bcf4e838da6d2fa19a8483c/users/a544142f73854f2c87c8c26841456074/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:35.874536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f093a53-7d3c-45f9-a89f-6f0916992964 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:35.880420 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0d7c8-0f90-447a-a51d-684caad87415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:35.881978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0d7c8-0f90-447a-a51d-684caad87415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:35.886057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0d7c8-0f90-447a-a51d-684caad87415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:35.887602 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0d7c8-0f90-447a-a51d-684caad87415 admin admin] PATH_INFO: `/v3/projects/4bbba53d79b14d64a04ab0be9562bd92` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:35.892531 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0d7c8-0f90-447a-a51d-684caad87415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bbba53d79b14d64a04ab0be9562bd92)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:35.907336 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0d7c8-0f90-447a-a51d-684caad87415 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:36.041004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fae0d7c8-0f90-447a-a51d-684caad87415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bbba53d79b14d64a04ab0be9562bd92', 'request_id': 'req-fae0d7c8-0f90-447a-a51d-684caad87415'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:36.044192 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f093a53-7d3c-45f9-a89f-6f0916992964 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:36.056694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fae0d7c8-0f90-447a-a51d-684caad87415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bbba53d79b14d64a04ab0be9562bd92', 'request_id': 'req-fae0d7c8-0f90-447a-a51d-684caad87415'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:36.071556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fae0d7c8-0f90-447a-a51d-684caad87415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bbba53d79b14d64a04ab0be9562bd92', 'request_id': 'req-fae0d7c8-0f90-447a-a51d-684caad87415'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:36.108974 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fae0d7c8-0f90-447a-a51d-684caad87415 admin admin] The token cache is being invalidate because project 4bbba53d79b14d64a04ab0be9562bd92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:36.109359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fae0d7c8-0f90-447a-a51d-684caad87415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:36.110130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fae0d7c8-0f90-447a-a51d-684caad87415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:36.114031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 893/1799] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:35 2022] DELETE /identity/v3/projects/4bbba53d79b14d64a04ab0be9562bd92 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:36.119221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0d7c8-0f90-447a-a51d-684caad87415 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:36.122515 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fae0d7c8-0f90-447a-a51d-684caad87415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:36.140871 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:36.165912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eeff7d-7c1f-48ee-bb56-4be1ed364778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:36.166531 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eeff7d-7c1f-48ee-bb56-4be1ed364778 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:36.166959 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eeff7d-7c1f-48ee-bb56-4be1ed364778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:36.167242 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eeff7d-7c1f-48ee-bb56-4be1ed364778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:36.167984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eeff7d-7c1f-48ee-bb56-4be1ed364778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:36.170495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eeff7d-7c1f-48ee-bb56-4be1ed364778 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:36.181751 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 907/1800] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:36.183813 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3644ff-88e8-428e-8e03-5bb8b7536b69 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:36.184141 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3644ff-88e8-428e-8e03-5bb8b7536b69 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:36.184548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3644ff-88e8-428e-8e03-5bb8b7536b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:36.211551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ba3644ff-88e8-428e-8e03-5bb8b7536b69 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:36.212969 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ba3644ff-88e8-428e-8e03-5bb8b7536b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:36.221444 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:36.229183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 908/1801] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:36.230215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3644ff-88e8-428e-8e03-5bb8b7536b69 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:36.231949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ba3644ff-88e8-428e-8e03-5bb8b7536b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:36.249633 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3644ff-88e8-428e-8e03-5bb8b7536b69 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:36.266321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604ba2f-e3e7-44aa-b4f3-c4ef3fa5e29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:36.267088 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604ba2f-e3e7-44aa-b4f3-c4ef3fa5e29d admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:36.267443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604ba2f-e3e7-44aa-b4f3-c4ef3fa5e29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:36.268077 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604ba2f-e3e7-44aa-b4f3-c4ef3fa5e29d admin admin] PATH_INFO: `/v3/projects/db879a8f7bcf4e838da6d2fa19a8483c/users/a544142f73854f2c87c8c26841456074/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:36.272120 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604ba2f-e3e7-44aa-b4f3-c4ef3fa5e29d admin admin] RBAC: Authorizing `identity:create_grant(project_id=db879a8f7bcf4e838da6d2fa19a8483c, user_id=a544142f73854f2c87c8c26841456074, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:36.274853 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604ba2f-e3e7-44aa-b4f3-c4ef3fa5e29d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:36.308405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 909/1802] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:36 2022] PUT /identity/v3/projects/db879a8f7bcf4e838da6d2fa19a8483c/users/a544142f73854f2c87c8c26841456074/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:36.308405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc455155-6741-46c0-af3f-c2a47c680a08 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:36.308405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc455155-6741-46c0-af3f-c2a47c680a08 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:36.308405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc455155-6741-46c0-af3f-c2a47c680a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:36.317104 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0d7c8-0f90-447a-a51d-684caad87415 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:36.333304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-dc455155-6741-46c0-af3f-c2a47c680a08 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:36.334512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-dc455155-6741-46c0-af3f-c2a47c680a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:36.409336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 910/1803] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:36.411746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205f5a7-83df-4d6b-8e83-6b2e9608f8bf None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:36.412233 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205f5a7-83df-4d6b-8e83-6b2e9608f8bf None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:36.412803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205f5a7-83df-4d6b-8e83-6b2e9608f8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:36.432404 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5df71-2f46-422d-be4a-db18ed62a73b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:36.433033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5df71-2f46-422d-be4a-db18ed62a73b service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:36.433313 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5df71-2f46-422d-be4a-db18ed62a73b service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:36.433585 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5df71-2f46-422d-be4a-db18ed62a73b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:36.436483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0dc5df71-2f46-422d-be4a-db18ed62a73b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:36.449723 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f205f5a7-83df-4d6b-8e83-6b2e9608f8bf None None] MFA Rules not processed for user `a544142f73854f2c87c8c26841456074`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:36.451394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f205f5a7-83df-4d6b-8e83-6b2e9608f8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:36.514464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5df71-2f46-422d-be4a-db18ed62a73b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:36.516180 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5df71-2f46-422d-be4a-db18ed62a73b service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:36.526193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 894/1804] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:22:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 12:22:36.527030 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5df71-2f46-422d-be4a-db18ed62a73b service swift] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:36.540060 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5df71-2f46-422d-be4a-db18ed62a73b service swift] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:36.546484 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae618cc-51a3-4c26-822f-2c606aa30f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:36.547242 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae618cc-51a3-4c26-822f-2c606aa30f59 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:36.548754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae618cc-51a3-4c26-822f-2c606aa30f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:36.549145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae618cc-51a3-4c26-822f-2c606aa30f59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:36.550304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae618cc-51a3-4c26-822f-2c606aa30f59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:36.550817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae618cc-51a3-4c26-822f-2c606aa30f59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:36.553364 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae618cc-51a3-4c26-822f-2c606aa30f59 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:36.586307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 911/1805] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:36.597039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 895/1806] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:36.598694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205f5a7-83df-4d6b-8e83-6b2e9608f8bf None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:36.601091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f205f5a7-83df-4d6b-8e83-6b2e9608f8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:36.620294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae618cc-51a3-4c26-822f-2c606aa30f59 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:36.669310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae618cc-51a3-4c26-822f-2c606aa30f59 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:36.684875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c06a84-926c-411f-a8db-45abc47072cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:36.685504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c06a84-926c-411f-a8db-45abc47072cc admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:36.685870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c06a84-926c-411f-a8db-45abc47072cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:36.686039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c06a84-926c-411f-a8db-45abc47072cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:36.686679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c06a84-926c-411f-a8db-45abc47072cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:36.688802 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c06a84-926c-411f-a8db-45abc47072cc admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:36.716489 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:36.725094 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205f5a7-83df-4d6b-8e83-6b2e9608f8bf None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:36.737697 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506bbe8-84f1-4630-9e35-841831c99760 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:36.742439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506bbe8-84f1-4630-9e35-841831c99760 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:36.743050 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506bbe8-84f1-4630-9e35-841831c99760 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:36.743401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506bbe8-84f1-4630-9e35-841831c99760 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:36.754872 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506bbe8-84f1-4630-9e35-841831c99760 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:36.757036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506bbe8-84f1-4630-9e35-841831c99760 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:36.770169 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 896/1807] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:36 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:36.771212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c06a84-926c-411f-a8db-45abc47072cc admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:36.772292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 912/1808] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:36 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:36.843073 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:36.870539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c06a84-926c-411f-a8db-45abc47072cc admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:36.875360 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e1a08-f83d-45b2-81c4-74b2b7b22d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:36.876078 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e1a08-f83d-45b2-81c4-74b2b7b22d75 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:36.876362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e1a08-f83d-45b2-81c4-74b2b7b22d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:36.876633 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e1a08-f83d-45b2-81c4-74b2b7b22d75 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:36.877343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e1a08-f83d-45b2-81c4-74b2b7b22d75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:36.879188 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e1a08-f83d-45b2-81c4-74b2b7b22d75 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:36.907627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 897/1809] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:36 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:36.912139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506bbe8-84f1-4630-9e35-841831c99760 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:36.924848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506bbe8-84f1-4630-9e35-841831c99760 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:36.929641 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a37ba0-a98d-4aee-9e88-b546e9e19942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:36.930653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a37ba0-a98d-4aee-9e88-b546e9e19942 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:36.930653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a37ba0-a98d-4aee-9e88-b546e9e19942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:36.930851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a37ba0-a98d-4aee-9e88-b546e9e19942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:36.931444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a37ba0-a98d-4aee-9e88-b546e9e19942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:36.933094 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a37ba0-a98d-4aee-9e88-b546e9e19942 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:36.940693 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e1a08-f83d-45b2-81c4-74b2b7b22d75 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:36.944073 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 913/1810] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:36.947860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a37ba0-a98d-4aee-9e88-b546e9e19942 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:36.953243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e1a08-f83d-45b2-81c4-74b2b7b22d75 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:36.958823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a37ba0-a98d-4aee-9e88-b546e9e19942 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:36.959863 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bef85-881f-4021-8fd0-b7734270f6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:36.960593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bef85-881f-4021-8fd0-b7734270f6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:36.961095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bef85-881f-4021-8fd0-b7734270f6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:36.961594 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bef85-881f-4021-8fd0-b7734270f6cd admin admin] PATH_INFO: `/v3/users/5c87a8fd64e84dce9a3198bbaf1e5984` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:36.962926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bef85-881f-4021-8fd0-b7734270f6cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c87a8fd64e84dce9a3198bbaf1e5984)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:36.963364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f13b84-be59-4fee-9fc0-a9f430442a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:36.963997 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f13b84-be59-4fee-9fc0-a9f430442a48 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:36.964279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f13b84-be59-4fee-9fc0-a9f430442a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:36.964611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f13b84-be59-4fee-9fc0-a9f430442a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:36.965854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bef85-881f-4021-8fd0-b7734270f6cd admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:36.965967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f13b84-be59-4fee-9fc0-a9f430442a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:36.967438 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f13b84-be59-4fee-9fc0-a9f430442a48 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:36.981350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 914/1811] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:36.982454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f13b84-be59-4fee-9fc0-a9f430442a48 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:36.994417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f13b84-be59-4fee-9fc0-a9f430442a48 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:36.999384 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a01f8-aeea-463d-8ecc-636064a526c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:37.000085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a01f8-aeea-463d-8ecc-636064a526c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:37.000370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a01f8-aeea-463d-8ecc-636064a526c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:37.000647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a01f8-aeea-463d-8ecc-636064a526c2 admin admin] PATH_INFO: `/v3/projects/c907748be7a844fd89b21f031c430873/users/07df5f30aa6a4351aadc03a572238aea/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:37.020048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a01f8-aeea-463d-8ecc-636064a526c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c907748be7a844fd89b21f031c430873, user_id=07df5f30aa6a4351aadc03a572238aea, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:37.022718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a01f8-aeea-463d-8ecc-636064a526c2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:37.037231 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 915/1812] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:36 2022] PUT /identity/v3/projects/c907748be7a844fd89b21f031c430873/users/07df5f30aa6a4351aadc03a572238aea/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:37.038142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-264bef85-881f-4021-8fd0-b7734270f6cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c87a8fd64e84dce9a3198bbaf1e5984'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:37.045813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-264bef85-881f-4021-8fd0-b7734270f6cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c87a8fd64e84dce9a3198bbaf1e5984'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:37.046617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-264bef85-881f-4021-8fd0-b7734270f6cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c87a8fd64e84dce9a3198bbaf1e5984'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:37.047315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-264bef85-881f-4021-8fd0-b7734270f6cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c87a8fd64e84dce9a3198bbaf1e5984'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:37.049197 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18532f-bdd9-4681-bc43-581d785b73b8 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:37.049471 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18532f-bdd9-4681-bc43-581d785b73b8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:37.049747 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18532f-bdd9-4681-bc43-581d785b73b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:37.055892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-264bef85-881f-4021-8fd0-b7734270f6cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c87a8fd64e84dce9a3198bbaf1e5984'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:37.075839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 898/1813] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:36 2022] DELETE /identity/v3/users/5c87a8fd64e84dce9a3198bbaf1e5984 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:37.079980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f22016-1db3-4c16-bf58-f6686acbef61 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:37.080264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f22016-1db3-4c16-bf58-f6686acbef61 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:37.080382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-2c18532f-bdd9-4681-bc43-581d785b73b8 None None] MFA Rules not processed for user `07df5f30aa6a4351aadc03a572238aea`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:37.080528 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f22016-1db3-4c16-bf58-f6686acbef61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:37.081865 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2c18532f-bdd9-4681-bc43-581d785b73b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:37.100098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-87f22016-1db3-4c16-bf58-f6686acbef61 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:37.101294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-87f22016-1db3-4c16-bf58-f6686acbef61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:37.196642 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 916/1814] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:22:37.199421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18532f-bdd9-4681-bc43-581d785b73b8 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:37.201762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2c18532f-bdd9-4681-bc43-581d785b73b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:37.209507 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 899/1815] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:37.217333 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18532f-bdd9-4681-bc43-581d785b73b8 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:37.219340 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f22016-1db3-4c16-bf58-f6686acbef61 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:37.220871 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-87f22016-1db3-4c16-bf58-f6686acbef61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:37.222792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864da77d-0258-42ec-afe8-5c4baf51e96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:37.224126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864da77d-0258-42ec-afe8-5c4baf51e96b admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:37.224419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864da77d-0258-42ec-afe8-5c4baf51e96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:37.224697 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864da77d-0258-42ec-afe8-5c4baf51e96b admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:37.225340 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864da77d-0258-42ec-afe8-5c4baf51e96b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:37.228252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864da77d-0258-42ec-afe8-5c4baf51e96b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:37.272196 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:37.275492 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 917/1816] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:37 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:22:37.276545 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864da77d-0258-42ec-afe8-5c4baf51e96b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:37.298123 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f22016-1db3-4c16-bf58-f6686acbef61 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:37.348235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864da77d-0258-42ec-afe8-5c4baf51e96b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:37.353090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522acf92-77ef-4ba9-ab9a-5dfbbe4cb316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:37.353823 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522acf92-77ef-4ba9-ab9a-5dfbbe4cb316 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:37.354107 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522acf92-77ef-4ba9-ab9a-5dfbbe4cb316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:37.354379 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522acf92-77ef-4ba9-ab9a-5dfbbe4cb316 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:37.355021 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522acf92-77ef-4ba9-ab9a-5dfbbe4cb316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:37.357040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522acf92-77ef-4ba9-ab9a-5dfbbe4cb316 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:37.358361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f719a-ba9e-46f5-9193-b552b8af8e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:37.359048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f719a-ba9e-46f5-9193-b552b8af8e7b service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:37.359429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f719a-ba9e-46f5-9193-b552b8af8e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:37.359796 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f719a-ba9e-46f5-9193-b552b8af8e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:37.370570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f719a-ba9e-46f5-9193-b552b8af8e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:37.372994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f719a-ba9e-46f5-9193-b552b8af8e7b service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:37.375811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:37.384572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 918/1817] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:37 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:37.387633 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522acf92-77ef-4ba9-ab9a-5dfbbe4cb316 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:37.388151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 900/1818] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:22:37.399321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522acf92-77ef-4ba9-ab9a-5dfbbe4cb316 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:37.409817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da739d75-a513-4289-a6ba-8bd99bada37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:37.411180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da739d75-a513-4289-a6ba-8bd99bada37a admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:37.411486 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da739d75-a513-4289-a6ba-8bd99bada37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:37.411908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da739d75-a513-4289-a6ba-8bd99bada37a admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:37.412691 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da739d75-a513-4289-a6ba-8bd99bada37a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:37.414382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da739d75-a513-4289-a6ba-8bd99bada37a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:37.437814 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f719a-ba9e-46f5-9193-b552b8af8e7b service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:37.450152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 919/1819] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:22:37.455812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f719a-ba9e-46f5-9193-b552b8af8e7b service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:37.463127 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca8077-6d81-425e-be8a-ded77ca27f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:37.463949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca8077-6d81-425e-be8a-ded77ca27f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:37.464229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca8077-6d81-425e-be8a-ded77ca27f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:37.464499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca8077-6d81-425e-be8a-ded77ca27f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:37.465797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ca8077-6d81-425e-be8a-ded77ca27f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:37.467431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ca8077-6d81-425e-be8a-ded77ca27f67 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:37.481228 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 901/1820] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:37.485686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da739d75-a513-4289-a6ba-8bd99bada37a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:37.513316 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da739d75-a513-4289-a6ba-8bd99bada37a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:37.525309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162981e-e84d-457e-8c42-531cb6ac1025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:37.526106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162981e-e84d-457e-8c42-531cb6ac1025 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:37.526849 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162981e-e84d-457e-8c42-531cb6ac1025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:37.527337 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162981e-e84d-457e-8c42-531cb6ac1025 admin admin] PATH_INFO: `/v3/projects/bd20bfdd981540c79bf861e529dbe70f/users/73cc6b4540b94e03a32695020f6e1337/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:37.544113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162981e-e84d-457e-8c42-531cb6ac1025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd20bfdd981540c79bf861e529dbe70f, user_id=73cc6b4540b94e03a32695020f6e1337, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:37.546777 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162981e-e84d-457e-8c42-531cb6ac1025 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:37.561496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 920/1821] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:37 2022] PUT /identity/v3/projects/bd20bfdd981540c79bf861e529dbe70f/users/73cc6b4540b94e03a32695020f6e1337/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:37.571659 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83250a-ebdc-43ba-9d66-0a4aa146bd25 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:37.572053 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83250a-ebdc-43ba-9d66-0a4aa146bd25 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:37.572390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83250a-ebdc-43ba-9d66-0a4aa146bd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:37.601026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-eb83250a-ebdc-43ba-9d66-0a4aa146bd25 None None] MFA Rules not processed for user `73cc6b4540b94e03a32695020f6e1337`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:37.602401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-eb83250a-ebdc-43ba-9d66-0a4aa146bd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:37.668146 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162981e-e84d-457e-8c42-531cb6ac1025 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:37.670180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b162981e-e84d-457e-8c42-531cb6ac1025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:37.670649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 902/1822] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:37.675147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83250a-ebdc-43ba-9d66-0a4aa146bd25 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:37.747651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162981e-e84d-457e-8c42-531cb6ac1025 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:37.748837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83250a-ebdc-43ba-9d66-0a4aa146bd25 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:37.752677 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f5068-c6b7-4ff7-9360-021b108886d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:37.753370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f5068-c6b7-4ff7-9360-021b108886d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:37.753519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03f50b-4e90-449f-b92b-565ca9b0dc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:37.753772 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f5068-c6b7-4ff7-9360-021b108886d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:37.754073 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f5068-c6b7-4ff7-9360-021b108886d3 admin admin] PATH_INFO: `/v3/projects/4305357752b24eb5a88a1ac84f506dd7` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:37.754178 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03f50b-4e90-449f-b92b-565ca9b0dc70 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:37.754418 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03f50b-4e90-449f-b92b-565ca9b0dc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:37.754690 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03f50b-4e90-449f-b92b-565ca9b0dc70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:37.755184 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f5068-c6b7-4ff7-9360-021b108886d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4305357752b24eb5a88a1ac84f506dd7)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:37.755351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03f50b-4e90-449f-b92b-565ca9b0dc70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:37.757165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f5068-c6b7-4ff7-9360-021b108886d3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:37.757291 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03f50b-4e90-449f-b92b-565ca9b0dc70 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:37.784585 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 903/1823] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:37 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:37.789054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03f50b-4e90-449f-b92b-565ca9b0dc70 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:37.794024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:37.806609 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-761f5068-c6b7-4ff7-9360-021b108886d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4305357752b24eb5a88a1ac84f506dd7', 'request_id': 'req-761f5068-c6b7-4ff7-9360-021b108886d3'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:37.814569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-761f5068-c6b7-4ff7-9360-021b108886d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4305357752b24eb5a88a1ac84f506dd7', 'request_id': 'req-761f5068-c6b7-4ff7-9360-021b108886d3'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:37.820883 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-761f5068-c6b7-4ff7-9360-021b108886d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4305357752b24eb5a88a1ac84f506dd7', 'request_id': 'req-761f5068-c6b7-4ff7-9360-021b108886d3'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:37.831415 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-761f5068-c6b7-4ff7-9360-021b108886d3 admin admin] The token cache is being invalidate because project 4305357752b24eb5a88a1ac84f506dd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:37.831719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-761f5068-c6b7-4ff7-9360-021b108886d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:37.832389 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-761f5068-c6b7-4ff7-9360-021b108886d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:37.833911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 921/1824] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:37 2022] DELETE /identity/v3/projects/4305357752b24eb5a88a1ac84f506dd7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:37.838143 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f5068-c6b7-4ff7-9360-021b108886d3 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:37.840453 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-761f5068-c6b7-4ff7-9360-021b108886d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:37.855486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03f50b-4e90-449f-b92b-565ca9b0dc70 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:37.899782 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:37.906718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f5068-c6b7-4ff7-9360-021b108886d3 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:37.912953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ad98e-59e2-4cc6-9fde-42b9b131106e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:37.913525 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ad98e-59e2-4cc6-9fde-42b9b131106e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:37.913811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ad98e-59e2-4cc6-9fde-42b9b131106e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:37.914086 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ad98e-59e2-4cc6-9fde-42b9b131106e admin admin] PATH_INFO: `/v3/users/a544142f73854f2c87c8c26841456074` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:37.915253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ad98e-59e2-4cc6-9fde-42b9b131106e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a544142f73854f2c87c8c26841456074)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:37.915889 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f3b8e-f694-4736-a4c9-41d6472394fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:37.916500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f3b8e-f694-4736-a4c9-41d6472394fc admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:37.916784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f3b8e-f694-4736-a4c9-41d6472394fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:37.917062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f3b8e-f694-4736-a4c9-41d6472394fc admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:37.917227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ad98e-59e2-4cc6-9fde-42b9b131106e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:37.917691 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f3b8e-f694-4736-a4c9-41d6472394fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:37.920246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f3b8e-f694-4736-a4c9-41d6472394fc admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:37.941048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 904/1825] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:37 2022] POST /identity/v3/users => generated 295 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:22:37.945309 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f3b8e-f694-4736-a4c9-41d6472394fc admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:37.947099 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e62f3b8e-f694-4736-a4c9-41d6472394fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:37.959313 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f3b8e-f694-4736-a4c9-41d6472394fc admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:37.964033 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-010ad98e-59e2-4cc6-9fde-42b9b131106e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a544142f73854f2c87c8c26841456074'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:37.964153 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e11806-7266-46f9-8202-dc54c03ede36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:37.964715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e11806-7266-46f9-8202-dc54c03ede36 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:37.965523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e11806-7266-46f9-8202-dc54c03ede36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:37.965799 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e11806-7266-46f9-8202-dc54c03ede36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:37.966439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e11806-7266-46f9-8202-dc54c03ede36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:37.968185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e11806-7266-46f9-8202-dc54c03ede36 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:37.972181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-010ad98e-59e2-4cc6-9fde-42b9b131106e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a544142f73854f2c87c8c26841456074'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:37.972882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-010ad98e-59e2-4cc6-9fde-42b9b131106e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a544142f73854f2c87c8c26841456074'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:37.973554 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-010ad98e-59e2-4cc6-9fde-42b9b131106e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a544142f73854f2c87c8c26841456074'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:37.978343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 905/1826] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:22:37.981981 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-010ad98e-59e2-4cc6-9fde-42b9b131106e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a544142f73854f2c87c8c26841456074'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:37.983900 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e11806-7266-46f9-8202-dc54c03ede36 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:37.985574 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-39e11806-7266-46f9-8202-dc54c03ede36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:37.998116 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e11806-7266-46f9-8202-dc54c03ede36 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:38.000082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 922/1827] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:37 2022] DELETE /identity/v3/users/a544142f73854f2c87c8c26841456074 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:38.056898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2726ce-8459-46ca-9331-feb0d5991b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:38.057553 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2726ce-8459-46ca-9331-feb0d5991b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:38.057832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2726ce-8459-46ca-9331-feb0d5991b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:38.058098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2726ce-8459-46ca-9331-feb0d5991b6b admin admin] PATH_INFO: `/v3/projects/d97ee743a9034f3a9cf2c15db0b7725a/users/e89eb6b8c393447eae4b323b8e764af7/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:38.075283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2726ce-8459-46ca-9331-feb0d5991b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d97ee743a9034f3a9cf2c15db0b7725a, user_id=e89eb6b8c393447eae4b323b8e764af7, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:38.078550 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2726ce-8459-46ca-9331-feb0d5991b6b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:38.092031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 906/1828] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:37 2022] PUT /identity/v3/projects/d97ee743a9034f3a9cf2c15db0b7725a/users/e89eb6b8c393447eae4b323b8e764af7/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:38.096144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ad98e-59e2-4cc6-9fde-42b9b131106e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:38.106942 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ad98e-59e2-4cc6-9fde-42b9b131106e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:38.169105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94490a-99d3-4dfd-9783-b80d167790f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:38.169646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94490a-99d3-4dfd-9783-b80d167790f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:38.169927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94490a-99d3-4dfd-9783-b80d167790f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:38.170198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94490a-99d3-4dfd-9783-b80d167790f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:38.170842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94490a-99d3-4dfd-9783-b80d167790f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:38.172482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94490a-99d3-4dfd-9783-b80d167790f6 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:38.182309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 923/1829] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:22:38.187013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2726ce-8459-46ca-9331-feb0d5991b6b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:38.198509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2726ce-8459-46ca-9331-feb0d5991b6b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:38.204032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce9646-5d0a-4919-8e77-af0ebb65a90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:38.204763 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce9646-5d0a-4919-8e77-af0ebb65a90b admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:38.205129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce9646-5d0a-4919-8e77-af0ebb65a90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:38.205478 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce9646-5d0a-4919-8e77-af0ebb65a90b admin admin] PATH_INFO: `/v3/projects/d97ee743a9034f3a9cf2c15db0b7725a/users/e89eb6b8c393447eae4b323b8e764af7/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:38.208231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce9646-5d0a-4919-8e77-af0ebb65a90b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d97ee743a9034f3a9cf2c15db0b7725a, user_id=e89eb6b8c393447eae4b323b8e764af7, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:38.210841 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce9646-5d0a-4919-8e77-af0ebb65a90b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:38.227313 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:38.243617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 907/1830] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:38 2022] PUT /identity/v3/projects/d97ee743a9034f3a9cf2c15db0b7725a/users/e89eb6b8c393447eae4b323b8e764af7/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:38.252779 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba82597-f21d-4091-a672-b21e58d4c469 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:38.253081 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba82597-f21d-4091-a672-b21e58d4c469 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:38.253338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba82597-f21d-4091-a672-b21e58d4c469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:38.282300 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce9646-5d0a-4919-8e77-af0ebb65a90b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:38.284312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b7ce9646-5d0a-4919-8e77-af0ebb65a90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:38.324239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8ba82597-f21d-4091-a672-b21e58d4c469 None None] MFA Rules not processed for user `e89eb6b8c393447eae4b323b8e764af7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:38.326650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8ba82597-f21d-4091-a672-b21e58d4c469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:38.401087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce9646-5d0a-4919-8e77-af0ebb65a90b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:38.406888 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cdfc2-7a87-4625-adf6-032331a3245e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:38.407361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cdfc2-7a87-4625-adf6-032331a3245e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:38.407669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cdfc2-7a87-4625-adf6-032331a3245e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:38.407955 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cdfc2-7a87-4625-adf6-032331a3245e admin admin] PATH_INFO: `/v3/projects/db879a8f7bcf4e838da6d2fa19a8483c` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:38.411555 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:38.416258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82cdfc2-7a87-4625-adf6-032331a3245e admin admin] RBAC: Authorizing `identity:delete_project(project_id=db879a8f7bcf4e838da6d2fa19a8483c)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:38.420425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82cdfc2-7a87-4625-adf6-032331a3245e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:38.423759 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 924/1831] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:38.433237 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba82597-f21d-4091-a672-b21e58d4c469 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:38.434782 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8ba82597-f21d-4091-a672-b21e58d4c469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:38.471384 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a82cdfc2-7a87-4625-adf6-032331a3245e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db879a8f7bcf4e838da6d2fa19a8483c', 'request_id': 'req-a82cdfc2-7a87-4625-adf6-032331a3245e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:38.480241 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a82cdfc2-7a87-4625-adf6-032331a3245e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db879a8f7bcf4e838da6d2fa19a8483c', 'request_id': 'req-a82cdfc2-7a87-4625-adf6-032331a3245e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:38.487065 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a82cdfc2-7a87-4625-adf6-032331a3245e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db879a8f7bcf4e838da6d2fa19a8483c', 'request_id': 'req-a82cdfc2-7a87-4625-adf6-032331a3245e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:38.496615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba82597-f21d-4091-a672-b21e58d4c469 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:38.497797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a82cdfc2-7a87-4625-adf6-032331a3245e admin admin] The token cache is being invalidate because project db879a8f7bcf4e838da6d2fa19a8483c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:38.498141 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a82cdfc2-7a87-4625-adf6-032331a3245e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:38.498794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a82cdfc2-7a87-4625-adf6-032331a3245e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:38.500373 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 908/1832] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:38 2022] DELETE /identity/v3/projects/db879a8f7bcf4e838da6d2fa19a8483c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:38.509291 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab8317-8162-47ac-ba20-9837763b2828 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:38.509582 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab8317-8162-47ac-ba20-9837763b2828 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:38.509853 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab8317-8162-47ac-ba20-9837763b2828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:38.537423 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-89ab8317-8162-47ac-ba20-9837763b2828 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:38.538943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-89ab8317-8162-47ac-ba20-9837763b2828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:38.568980 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ee90c-bfe3-4147-9e0a-f0d82557f118 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:38.569877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ee90c-bfe3-4147-9e0a-f0d82557f118 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:38.570325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ee90c-bfe3-4147-9e0a-f0d82557f118 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:38.570677 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ee90c-bfe3-4147-9e0a-f0d82557f118 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:38.572797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-805ee90c-bfe3-4147-9e0a-f0d82557f118 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:38.628864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 909/1833] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:38.636339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550e507-a0cf-4392-b72b-3256c400ff74 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:38.636632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550e507-a0cf-4392-b72b-3256c400ff74 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:38.636979 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550e507-a0cf-4392-b72b-3256c400ff74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:38.657430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9550e507-a0cf-4392-b72b-3256c400ff74 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:38.658642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9550e507-a0cf-4392-b72b-3256c400ff74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:38.664922 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ee90c-bfe3-4147-9e0a-f0d82557f118 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:38.666408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ee90c-bfe3-4147-9e0a-f0d82557f118 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:38.667540 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 910/1834] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:22:38.672577 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550e507-a0cf-4392-b72b-3256c400ff74 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:38.681640 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550e507-a0cf-4392-b72b-3256c400ff74 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:38.682703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 925/1835] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:22:38.687892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179243f1-a275-4b55-8bfb-2485f35dfc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:38.688449 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179243f1-a275-4b55-8bfb-2485f35dfc91 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:38.688735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179243f1-a275-4b55-8bfb-2485f35dfc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:38.689011 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179243f1-a275-4b55-8bfb-2485f35dfc91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:38.689660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179243f1-a275-4b55-8bfb-2485f35dfc91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:38.689967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179243f1-a275-4b55-8bfb-2485f35dfc91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:38.691619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179243f1-a275-4b55-8bfb-2485f35dfc91 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:38.707560 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 911/1836] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:22:38.709596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ee90c-bfe3-4147-9e0a-f0d82557f118 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:38.720434 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ee90c-bfe3-4147-9e0a-f0d82557f118 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:38.724452 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179243f1-a275-4b55-8bfb-2485f35dfc91 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:38.725337 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581487d6-400d-4fe7-9a8b-8e2d759cb719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:38.726056 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581487d6-400d-4fe7-9a8b-8e2d759cb719 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:38.726168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-179243f1-a275-4b55-8bfb-2485f35dfc91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:38.726345 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581487d6-400d-4fe7-9a8b-8e2d759cb719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:38.726618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581487d6-400d-4fe7-9a8b-8e2d759cb719 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:38.727254 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581487d6-400d-4fe7-9a8b-8e2d759cb719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:38.731715 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581487d6-400d-4fe7-9a8b-8e2d759cb719 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:38.738820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179243f1-a275-4b55-8bfb-2485f35dfc91 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:38.742616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b82d7-1094-4501-8e26-e71300601246 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:38.743714 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b82d7-1094-4501-8e26-e71300601246 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:38.743994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b82d7-1094-4501-8e26-e71300601246 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:38.744263 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b82d7-1094-4501-8e26-e71300601246 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:38.746978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-612b82d7-1094-4501-8e26-e71300601246 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:38.755743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:38.766113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 926/1837] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:38 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:38.770150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581487d6-400d-4fe7-9a8b-8e2d759cb719 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:38.812930 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b82d7-1094-4501-8e26-e71300601246 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:38.813973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:38.814372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b82d7-1094-4501-8e26-e71300601246 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:38.824271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 912/1838] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:22:38.836377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581487d6-400d-4fe7-9a8b-8e2d759cb719 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:38.841209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51bac6-3066-4d84-bce7-210b5a356e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:38.842013 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51bac6-3066-4d84-bce7-210b5a356e01 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:38.842293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51bac6-3066-4d84-bce7-210b5a356e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:38.842567 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51bac6-3066-4d84-bce7-210b5a356e01 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:38.843196 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51bac6-3066-4d84-bce7-210b5a356e01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:38.845285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51bac6-3066-4d84-bce7-210b5a356e01 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:38.866770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 927/1839] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:38 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:22:38.870980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b82d7-1094-4501-8e26-e71300601246 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:38.883287 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b82d7-1094-4501-8e26-e71300601246 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:38.888758 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99698085-3390-4222-82f2-fbebbab5a92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:38.889367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99698085-3390-4222-82f2-fbebbab5a92b admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:38.889650 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99698085-3390-4222-82f2-fbebbab5a92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:38.889954 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99698085-3390-4222-82f2-fbebbab5a92b admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:38.890597 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99698085-3390-4222-82f2-fbebbab5a92b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:38.892264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99698085-3390-4222-82f2-fbebbab5a92b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:38.903318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 913/1840] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:38.907407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51bac6-3066-4d84-bce7-210b5a356e01 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:38.917891 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51bac6-3066-4d84-bce7-210b5a356e01 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:38.922759 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196ae95-c520-4162-b2a0-c36e3c5b888c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:38.923297 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196ae95-c520-4162-b2a0-c36e3c5b888c admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:38.923739 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196ae95-c520-4162-b2a0-c36e3c5b888c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:38.924012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196ae95-c520-4162-b2a0-c36e3c5b888c admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:38.924653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196ae95-c520-4162-b2a0-c36e3c5b888c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:38.926233 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196ae95-c520-4162-b2a0-c36e3c5b888c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:38.940825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 928/1841] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:38.942028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99698085-3390-4222-82f2-fbebbab5a92b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:38.952523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99698085-3390-4222-82f2-fbebbab5a92b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:38.957190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5778b-00a9-421f-939d-eccb47bd6c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:38.957849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5778b-00a9-421f-939d-eccb47bd6c27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:38.958133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5778b-00a9-421f-939d-eccb47bd6c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:38.958409 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5778b-00a9-421f-939d-eccb47bd6c27 admin admin] PATH_INFO: `/v3/projects/45d44dd7707a45399e078892328d5e85/users/4e5fd1231b034e8798ea0071e0a71f21/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:38.973969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5778b-00a9-421f-939d-eccb47bd6c27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d44dd7707a45399e078892328d5e85, user_id=4e5fd1231b034e8798ea0071e0a71f21, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:38.976678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5778b-00a9-421f-939d-eccb47bd6c27 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:38.990199 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 914/1842] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:38 2022] PUT /identity/v3/projects/45d44dd7707a45399e078892328d5e85/users/4e5fd1231b034e8798ea0071e0a71f21/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:39.000574 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb4d0a-b14d-4c7e-9d4d-10f5e9518813 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:39.000892 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb4d0a-b14d-4c7e-9d4d-10f5e9518813 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:39.001167 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb4d0a-b14d-4c7e-9d4d-10f5e9518813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:39.031425 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fdeb4d0a-b14d-4c7e-9d4d-10f5e9518813 None None] MFA Rules not processed for user `4e5fd1231b034e8798ea0071e0a71f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:39.032727 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fdeb4d0a-b14d-4c7e-9d4d-10f5e9518813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:39.101109 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 929/1843] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:22:39.106855 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5778b-00a9-421f-939d-eccb47bd6c27 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:39.218187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5778b-00a9-421f-939d-eccb47bd6c27 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:39.222206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0280ddb-5e32-44bc-b2b9-89545b8e908e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:39.222859 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0280ddb-5e32-44bc-b2b9-89545b8e908e admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:39.223147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0280ddb-5e32-44bc-b2b9-89545b8e908e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:39.223491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0280ddb-5e32-44bc-b2b9-89545b8e908e admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:39.224324 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0280ddb-5e32-44bc-b2b9-89545b8e908e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:39.226757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0280ddb-5e32-44bc-b2b9-89545b8e908e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:39.302210 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb4d0a-b14d-4c7e-9d4d-10f5e9518813 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:39.302727 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 915/1844] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:39 2022] POST /identity/v3/projects => generated 383 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:39.316968 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb4d0a-b14d-4c7e-9d4d-10f5e9518813 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:39.432064 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163a2a4-18c3-45c4-88e2-ae7babacae01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:39.432807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163a2a4-18c3-45c4-88e2-ae7babacae01 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:39.433091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163a2a4-18c3-45c4-88e2-ae7babacae01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:39.433476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163a2a4-18c3-45c4-88e2-ae7babacae01 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:39.434979 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163a2a4-18c3-45c4-88e2-ae7babacae01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:39.439903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163a2a4-18c3-45c4-88e2-ae7babacae01 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:39.483635 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 930/1845] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:39 2022] POST /identity/v3/users => generated 298 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:39.487220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0280ddb-5e32-44bc-b2b9-89545b8e908e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:39.503153 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0280ddb-5e32-44bc-b2b9-89545b8e908e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:39.508695 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88390fca-116e-466b-b2a5-fe1ccfe20a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:39.509487 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88390fca-116e-466b-b2a5-fe1ccfe20a2d admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:39.509777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88390fca-116e-466b-b2a5-fe1ccfe20a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:39.510055 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88390fca-116e-466b-b2a5-fe1ccfe20a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:39.511514 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88390fca-116e-466b-b2a5-fe1ccfe20a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:39.515568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88390fca-116e-466b-b2a5-fe1ccfe20a2d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:39.537358 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 916/1846] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:39.543538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163a2a4-18c3-45c4-88e2-ae7babacae01 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:39.556341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163a2a4-18c3-45c4-88e2-ae7babacae01 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:39.561935 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946a1ac-ebc1-4716-9116-d1f6dcf5c306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:39.562599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946a1ac-ebc1-4716-9116-d1f6dcf5c306 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:39.562882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946a1ac-ebc1-4716-9116-d1f6dcf5c306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:39.563152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946a1ac-ebc1-4716-9116-d1f6dcf5c306 admin admin] PATH_INFO: `/v3/projects/0b0b610ee7404339b4c1c5d8bf60d456/users/da061bce65274d7fbbbe38c3b8ccde24/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:39.584104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946a1ac-ebc1-4716-9116-d1f6dcf5c306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b0b610ee7404339b4c1c5d8bf60d456, user_id=da061bce65274d7fbbbe38c3b8ccde24, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:39.587814 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946a1ac-ebc1-4716-9116-d1f6dcf5c306 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:39.618004 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 931/1847] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:39 2022] PUT /identity/v3/projects/0b0b610ee7404339b4c1c5d8bf60d456/users/da061bce65274d7fbbbe38c3b8ccde24/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:39.622659 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88390fca-116e-466b-b2a5-fe1ccfe20a2d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:39.642365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88390fca-116e-466b-b2a5-fe1ccfe20a2d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:39.695228 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:39.720575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66a08f-2aaf-484c-bf4c-4ae344099437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:39.721203 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66a08f-2aaf-484c-bf4c-4ae344099437 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:39.721482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66a08f-2aaf-484c-bf4c-4ae344099437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:39.721832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66a08f-2aaf-484c-bf4c-4ae344099437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:39.722488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66a08f-2aaf-484c-bf4c-4ae344099437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:39.724153 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66a08f-2aaf-484c-bf4c-4ae344099437 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:39.736997 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 917/1848] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:39.741397 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946a1ac-ebc1-4716-9116-d1f6dcf5c306 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:39.755953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946a1ac-ebc1-4716-9116-d1f6dcf5c306 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:39.764198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793da465-48aa-4e82-9094-fea8c5d75dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:39.764973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793da465-48aa-4e82-9094-fea8c5d75dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:39.765278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793da465-48aa-4e82-9094-fea8c5d75dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:39.765496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793da465-48aa-4e82-9094-fea8c5d75dc6 admin admin] PATH_INFO: `/v3/projects/0b0b610ee7404339b4c1c5d8bf60d456/users/da061bce65274d7fbbbe38c3b8ccde24/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:39.767932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793da465-48aa-4e82-9094-fea8c5d75dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b0b610ee7404339b4c1c5d8bf60d456, user_id=da061bce65274d7fbbbe38c3b8ccde24, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:39.774407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793da465-48aa-4e82-9094-fea8c5d75dc6 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:39.788691 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:39.796086 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 932/1849] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:39 2022] PUT /identity/v3/projects/0b0b610ee7404339b4c1c5d8bf60d456/users/da061bce65274d7fbbbe38c3b8ccde24/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:39.812705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f17fd5-736d-4524-a649-64e7b4ea56bb None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:39.812996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f17fd5-736d-4524-a649-64e7b4ea56bb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:39.813606 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f17fd5-736d-4524-a649-64e7b4ea56bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:39.868899 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a1f17fd5-736d-4524-a649-64e7b4ea56bb None None] MFA Rules not processed for user `da061bce65274d7fbbbe38c3b8ccde24`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:39.870441 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a1f17fd5-736d-4524-a649-64e7b4ea56bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:39.966241 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 918/1850] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:39.977308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793da465-48aa-4e82-9094-fea8c5d75dc6 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:39.979194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-793da465-48aa-4e82-9094-fea8c5d75dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:40.044687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793da465-48aa-4e82-9094-fea8c5d75dc6 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:40.049292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02c9bc-db7c-4d83-8283-25662e31f0a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:40.049917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02c9bc-db7c-4d83-8283-25662e31f0a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:40.050190 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02c9bc-db7c-4d83-8283-25662e31f0a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:40.050488 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02c9bc-db7c-4d83-8283-25662e31f0a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:40.112509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02c9bc-db7c-4d83-8283-25662e31f0a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:40.113943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02c9bc-db7c-4d83-8283-25662e31f0a6 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:40.123356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 933/1851] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:44.261694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f17fd5-736d-4524-a649-64e7b4ea56bb None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:44.263324 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a1f17fd5-736d-4524-a649-64e7b4ea56bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:44.333483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f17fd5-736d-4524-a649-64e7b4ea56bb None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:44.340404 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d39bf6-1bbe-4cd6-a33a-39eb812c7c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:44.341101 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d39bf6-1bbe-4cd6-a33a-39eb812c7c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:44.341389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d39bf6-1bbe-4cd6-a33a-39eb812c7c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:44.341771 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d39bf6-1bbe-4cd6-a33a-39eb812c7c65 admin admin] PATH_INFO: `/v3/users/07df5f30aa6a4351aadc03a572238aea` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:44.342972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d39bf6-1bbe-4cd6-a33a-39eb812c7c65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07df5f30aa6a4351aadc03a572238aea)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:44.344951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d39bf6-1bbe-4cd6-a33a-39eb812c7c65 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:44.409288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d6d39bf6-1bbe-4cd6-a33a-39eb812c7c65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07df5f30aa6a4351aadc03a572238aea'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:44.418308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d6d39bf6-1bbe-4cd6-a33a-39eb812c7c65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07df5f30aa6a4351aadc03a572238aea'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:44.419299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d6d39bf6-1bbe-4cd6-a33a-39eb812c7c65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07df5f30aa6a4351aadc03a572238aea'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:44.420054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d6d39bf6-1bbe-4cd6-a33a-39eb812c7c65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07df5f30aa6a4351aadc03a572238aea'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:44.427485 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d6d39bf6-1bbe-4cd6-a33a-39eb812c7c65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07df5f30aa6a4351aadc03a572238aea'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:44.433184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:44.450109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 919/1852] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:44 2022] DELETE /identity/v3/users/07df5f30aa6a4351aadc03a572238aea => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:44.456379 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254b06e-c7e5-49f9-9243-d689985e6011 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:44.456903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254b06e-c7e5-49f9-9243-d689985e6011 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:44.457237 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254b06e-c7e5-49f9-9243-d689985e6011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:44.478549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-7254b06e-c7e5-49f9-9243-d689985e6011 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:44.480089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7254b06e-c7e5-49f9-9243-d689985e6011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:44.626036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 934/1853] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:44.635431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d39bf6-1bbe-4cd6-a33a-39eb812c7c65 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:44.637820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d6d39bf6-1bbe-4cd6-a33a-39eb812c7c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:44.786418 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d39bf6-1bbe-4cd6-a33a-39eb812c7c65 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:44.791393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368c689-ef39-4d43-b0f4-48e159194cb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:44.792363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368c689-ef39-4d43-b0f4-48e159194cb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:44.792929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368c689-ef39-4d43-b0f4-48e159194cb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:44.793412 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368c689-ef39-4d43-b0f4-48e159194cb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:44.805739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368c689-ef39-4d43-b0f4-48e159194cb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:44.807666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368c689-ef39-4d43-b0f4-48e159194cb7 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:44.821833 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 920/1854] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:45.342719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254b06e-c7e5-49f9-9243-d689985e6011 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:45.344714 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7254b06e-c7e5-49f9-9243-d689985e6011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:45.365235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254b06e-c7e5-49f9-9243-d689985e6011 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:45.371535 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81942563-4d81-42b6-9a75-1dc90385a29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:45.372403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81942563-4d81-42b6-9a75-1dc90385a29d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:45.372843 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81942563-4d81-42b6-9a75-1dc90385a29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:45.373279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81942563-4d81-42b6-9a75-1dc90385a29d admin admin] PATH_INFO: `/v3/users/c90ad34bac7a433681256368c36126da` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:45.375685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81942563-4d81-42b6-9a75-1dc90385a29d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c90ad34bac7a433681256368c36126da)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:45.380080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81942563-4d81-42b6-9a75-1dc90385a29d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:45.487606 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-81942563-4d81-42b6-9a75-1dc90385a29d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c90ad34bac7a433681256368c36126da'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:45.491453 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:45.509728 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-81942563-4d81-42b6-9a75-1dc90385a29d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c90ad34bac7a433681256368c36126da'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:45.515929 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-81942563-4d81-42b6-9a75-1dc90385a29d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c90ad34bac7a433681256368c36126da'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:45.515929 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-81942563-4d81-42b6-9a75-1dc90385a29d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c90ad34bac7a433681256368c36126da'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:45.523239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-81942563-4d81-42b6-9a75-1dc90385a29d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c90ad34bac7a433681256368c36126da'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:45.554864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 935/1855] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:45 2022] DELETE /identity/v3/users/c90ad34bac7a433681256368c36126da => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:45.560009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8aa964-e2cf-4363-853c-ba25581be812 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:45.560583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8aa964-e2cf-4363-853c-ba25581be812 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:45.560759 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8aa964-e2cf-4363-853c-ba25581be812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:45.618954 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2a8aa964-e2cf-4363-853c-ba25581be812 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:45.622087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2a8aa964-e2cf-4363-853c-ba25581be812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:45.672666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81942563-4d81-42b6-9a75-1dc90385a29d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:45.674499 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-81942563-4d81-42b6-9a75-1dc90385a29d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:45.732163 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 921/1856] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:22:45.743824 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8aa964-e2cf-4363-853c-ba25581be812 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:45.756469 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2a8aa964-e2cf-4363-853c-ba25581be812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:45.862279 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8aa964-e2cf-4363-853c-ba25581be812 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:45.867405 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280db612-e877-4b29-91ed-758eec9f0835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:45.868209 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280db612-e877-4b29-91ed-758eec9f0835 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:45.869081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280db612-e877-4b29-91ed-758eec9f0835 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:45.869436 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280db612-e877-4b29-91ed-758eec9f0835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:45.880850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81942563-4d81-42b6-9a75-1dc90385a29d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:45.883220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280db612-e877-4b29-91ed-758eec9f0835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:45.884813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280db612-e877-4b29-91ed-758eec9f0835 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:45.894150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605c405-a97d-481c-9ca9-40e7bede48ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:45.894810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605c405-a97d-481c-9ca9-40e7bede48ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:45.895099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605c405-a97d-481c-9ca9-40e7bede48ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:45.895375 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605c405-a97d-481c-9ca9-40e7bede48ab admin admin] PATH_INFO: `/v3/projects/c907748be7a844fd89b21f031c430873` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:45.897923 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 922/1857] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:22:45.923418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605c405-a97d-481c-9ca9-40e7bede48ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=c907748be7a844fd89b21f031c430873)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:45.925399 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605c405-a97d-481c-9ca9-40e7bede48ab admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:46.063803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4605c405-a97d-481c-9ca9-40e7bede48ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c907748be7a844fd89b21f031c430873', 'request_id': 'req-4605c405-a97d-481c-9ca9-40e7bede48ab'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:46.086626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4605c405-a97d-481c-9ca9-40e7bede48ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c907748be7a844fd89b21f031c430873', 'request_id': 'req-4605c405-a97d-481c-9ca9-40e7bede48ab'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:46.107184 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4605c405-a97d-481c-9ca9-40e7bede48ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c907748be7a844fd89b21f031c430873', 'request_id': 'req-4605c405-a97d-481c-9ca9-40e7bede48ab'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:46.123330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4605c405-a97d-481c-9ca9-40e7bede48ab admin admin] The token cache is being invalidate because project c907748be7a844fd89b21f031c430873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:46.123802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4605c405-a97d-481c-9ca9-40e7bede48ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:46.124648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4605c405-a97d-481c-9ca9-40e7bede48ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:46.127541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 936/1858] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:45 2022] DELETE /identity/v3/projects/c907748be7a844fd89b21f031c430873 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:46.131330 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280db612-e877-4b29-91ed-758eec9f0835 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:46.144688 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280db612-e877-4b29-91ed-758eec9f0835 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:46.218841 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:46.318933 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29dfa3-2204-4034-94ef-edb946606f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:46.321116 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29dfa3-2204-4034-94ef-edb946606f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:46.322414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29dfa3-2204-4034-94ef-edb946606f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:46.323822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29dfa3-2204-4034-94ef-edb946606f6b admin admin] PATH_INFO: `/v3/users/73cc6b4540b94e03a32695020f6e1337` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:46.326328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29dfa3-2204-4034-94ef-edb946606f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=73cc6b4540b94e03a32695020f6e1337)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:46.333363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29dfa3-2204-4034-94ef-edb946606f6b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:46.507213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2c29dfa3-2204-4034-94ef-edb946606f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73cc6b4540b94e03a32695020f6e1337'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:46.523362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2c29dfa3-2204-4034-94ef-edb946606f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73cc6b4540b94e03a32695020f6e1337'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:46.524404 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2c29dfa3-2204-4034-94ef-edb946606f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73cc6b4540b94e03a32695020f6e1337'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:46.525262 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2c29dfa3-2204-4034-94ef-edb946606f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73cc6b4540b94e03a32695020f6e1337'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:46.555392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2c29dfa3-2204-4034-94ef-edb946606f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73cc6b4540b94e03a32695020f6e1337'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:46.597130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 923/1859] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:46 2022] DELETE /identity/v3/users/73cc6b4540b94e03a32695020f6e1337 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:46.687887 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605c405-a97d-481c-9ca9-40e7bede48ab admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:46.690694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4605c405-a97d-481c-9ca9-40e7bede48ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:46.717699 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605c405-a97d-481c-9ca9-40e7bede48ab admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:46.862898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:46.873026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0c70f-acb3-412f-91da-d89c9fa00dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:46.874525 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0c70f-acb3-412f-91da-d89c9fa00dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:46.875097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0c70f-acb3-412f-91da-d89c9fa00dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:46.891038 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0c70f-acb3-412f-91da-d89c9fa00dd4 admin admin] PATH_INFO: `/v3/projects/99447455b6974f4888380e172e16c46e` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:46.894762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0c70f-acb3-412f-91da-d89c9fa00dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99447455b6974f4888380e172e16c46e)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:46.904313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0c70f-acb3-412f-91da-d89c9fa00dd4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:47.070063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-17f0c70f-acb3-412f-91da-d89c9fa00dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99447455b6974f4888380e172e16c46e', 'request_id': 'req-17f0c70f-acb3-412f-91da-d89c9fa00dd4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.087658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-17f0c70f-acb3-412f-91da-d89c9fa00dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99447455b6974f4888380e172e16c46e', 'request_id': 'req-17f0c70f-acb3-412f-91da-d89c9fa00dd4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.120202 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-17f0c70f-acb3-412f-91da-d89c9fa00dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99447455b6974f4888380e172e16c46e', 'request_id': 'req-17f0c70f-acb3-412f-91da-d89c9fa00dd4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.153972 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-17f0c70f-acb3-412f-91da-d89c9fa00dd4 admin admin] The token cache is being invalidate because project 99447455b6974f4888380e172e16c46e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:47.154975 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-17f0c70f-acb3-412f-91da-d89c9fa00dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.155979 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-17f0c70f-acb3-412f-91da-d89c9fa00dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.158301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 937/1860] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:46 2022] DELETE /identity/v3/projects/99447455b6974f4888380e172e16c46e => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:47.162581 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29dfa3-2204-4034-94ef-edb946606f6b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:47.164416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2c29dfa3-2204-4034-94ef-edb946606f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:47.180413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29dfa3-2204-4034-94ef-edb946606f6b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:47.257882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0c70f-acb3-412f-91da-d89c9fa00dd4 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:47.260051 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-17f0c70f-acb3-412f-91da-d89c9fa00dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:47.283262 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5cbb16-954c-4c5c-9575-6e594b4947c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:47.284028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cbb16-954c-4c5c-9575-6e594b4947c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:47.284373 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cbb16-954c-4c5c-9575-6e594b4947c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:47.284717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cbb16-954c-4c5c-9575-6e594b4947c2 admin admin] PATH_INFO: `/v3/users/fda45ac8ce02440ba5c6002bac3cd11d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:47.286232 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5cbb16-954c-4c5c-9575-6e594b4947c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fda45ac8ce02440ba5c6002bac3cd11d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:47.288486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5cbb16-954c-4c5c-9575-6e594b4947c2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:47.359302 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0c70f-acb3-412f-91da-d89c9fa00dd4 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:47.364492 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:47.366005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:47.366252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:47.366584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4 admin admin] PATH_INFO: `/v3/projects/bd20bfdd981540c79bf861e529dbe70f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:47.367821 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd20bfdd981540c79bf861e529dbe70f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:47.369783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:47.376401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8e5cbb16-954c-4c5c-9575-6e594b4947c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fda45ac8ce02440ba5c6002bac3cd11d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.385622 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:47.388923 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8e5cbb16-954c-4c5c-9575-6e594b4947c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fda45ac8ce02440ba5c6002bac3cd11d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.390031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8e5cbb16-954c-4c5c-9575-6e594b4947c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fda45ac8ce02440ba5c6002bac3cd11d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.390844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8e5cbb16-954c-4c5c-9575-6e594b4947c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fda45ac8ce02440ba5c6002bac3cd11d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.412526 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8e5cbb16-954c-4c5c-9575-6e594b4947c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fda45ac8ce02440ba5c6002bac3cd11d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.443035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 924/1861] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:47 2022] DELETE /identity/v3/users/fda45ac8ce02440ba5c6002bac3cd11d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:47.443628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd20bfdd981540c79bf861e529dbe70f', 'request_id': 'req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.452975 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd20bfdd981540c79bf861e529dbe70f', 'request_id': 'req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.460198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd20bfdd981540c79bf861e529dbe70f', 'request_id': 'req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.477111 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4 admin admin] The token cache is being invalidate because project bd20bfdd981540c79bf861e529dbe70f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:47.477456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.478153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.483572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 938/1862] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:47 2022] DELETE /identity/v3/projects/bd20bfdd981540c79bf861e529dbe70f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:47.484695 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5cbb16-954c-4c5c-9575-6e594b4947c2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:47.486163 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8e5cbb16-954c-4c5c-9575-6e594b4947c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:47.595752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5cbb16-954c-4c5c-9575-6e594b4947c2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:47.601881 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a22c3-a761-419b-a04d-433bc1d377fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:47.603139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a22c3-a761-419b-a04d-433bc1d377fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:47.603139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a22c3-a761-419b-a04d-433bc1d377fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:47.603139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a22c3-a761-419b-a04d-433bc1d377fc admin admin] PATH_INFO: `/v3/users/e89eb6b8c393447eae4b323b8e764af7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:47.604477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3a22c3-a761-419b-a04d-433bc1d377fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e89eb6b8c393447eae4b323b8e764af7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:47.607184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3a22c3-a761-419b-a04d-433bc1d377fc admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:47.670601 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-dd3a22c3-a761-419b-a04d-433bc1d377fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e89eb6b8c393447eae4b323b8e764af7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.679096 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-dd3a22c3-a761-419b-a04d-433bc1d377fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e89eb6b8c393447eae4b323b8e764af7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.679839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-dd3a22c3-a761-419b-a04d-433bc1d377fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e89eb6b8c393447eae4b323b8e764af7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.681017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-dd3a22c3-a761-419b-a04d-433bc1d377fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e89eb6b8c393447eae4b323b8e764af7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.692755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-dd3a22c3-a761-419b-a04d-433bc1d377fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e89eb6b8c393447eae4b323b8e764af7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:47.712654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 925/1863] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:47 2022] DELETE /identity/v3/users/e89eb6b8c393447eae4b323b8e764af7 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:48.000613 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:48.002990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:48.044938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:48.139915 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf492f2a-4afd-499c-8a2c-4ef6df2947f4 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:48.193308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a22c3-a761-419b-a04d-433bc1d377fc admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:48.195002 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-dd3a22c3-a761-419b-a04d-433bc1d377fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:48.304656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae82a8-ed1d-4542-a36c-ade4950269fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:48.308285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae82a8-ed1d-4542-a36c-ade4950269fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:48.308407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae82a8-ed1d-4542-a36c-ade4950269fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:48.308670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae82a8-ed1d-4542-a36c-ade4950269fa admin admin] PATH_INFO: `/v3/projects/4f146b5cf187406aa04b767aac3ce891` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:48.315531 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae82a8-ed1d-4542-a36c-ade4950269fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f146b5cf187406aa04b767aac3ce891)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:48.317547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae82a8-ed1d-4542-a36c-ade4950269fa admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:48.330668 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:48.367384 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a22c3-a761-419b-a04d-433bc1d377fc admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:48.373889 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d4b39-7d5e-4af8-95fb-d0fa68bd652c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:48.379836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d4b39-7d5e-4af8-95fb-d0fa68bd652c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:48.381792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d4b39-7d5e-4af8-95fb-d0fa68bd652c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:48.382231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d4b39-7d5e-4af8-95fb-d0fa68bd652c admin admin] PATH_INFO: `/v3/projects/d97ee743a9034f3a9cf2c15db0b7725a` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:48.385228 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d4b39-7d5e-4af8-95fb-d0fa68bd652c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d97ee743a9034f3a9cf2c15db0b7725a)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:48.387903 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d4b39-7d5e-4af8-95fb-d0fa68bd652c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:48.424274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c3ae82a8-ed1d-4542-a36c-ade4950269fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f146b5cf187406aa04b767aac3ce891', 'request_id': 'req-c3ae82a8-ed1d-4542-a36c-ade4950269fa'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:48.434019 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c3ae82a8-ed1d-4542-a36c-ade4950269fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f146b5cf187406aa04b767aac3ce891', 'request_id': 'req-c3ae82a8-ed1d-4542-a36c-ade4950269fa'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:48.440813 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c3ae82a8-ed1d-4542-a36c-ade4950269fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f146b5cf187406aa04b767aac3ce891', 'request_id': 'req-c3ae82a8-ed1d-4542-a36c-ade4950269fa'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:48.449396 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c35d4b39-7d5e-4af8-95fb-d0fa68bd652c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd97ee743a9034f3a9cf2c15db0b7725a', 'request_id': 'req-c35d4b39-7d5e-4af8-95fb-d0fa68bd652c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:48.454396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c3ae82a8-ed1d-4542-a36c-ade4950269fa admin admin] The token cache is being invalidate because project 4f146b5cf187406aa04b767aac3ce891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:48.454836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c3ae82a8-ed1d-4542-a36c-ade4950269fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:48.455717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c3ae82a8-ed1d-4542-a36c-ade4950269fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:48.457775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c35d4b39-7d5e-4af8-95fb-d0fa68bd652c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd97ee743a9034f3a9cf2c15db0b7725a', 'request_id': 'req-c35d4b39-7d5e-4af8-95fb-d0fa68bd652c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:48.458252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 939/1864] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:47 2022] DELETE /identity/v3/projects/4f146b5cf187406aa04b767aac3ce891 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:48.458831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae82a8-ed1d-4542-a36c-ade4950269fa admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:48.460589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c3ae82a8-ed1d-4542-a36c-ade4950269fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:48.470416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c35d4b39-7d5e-4af8-95fb-d0fa68bd652c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd97ee743a9034f3a9cf2c15db0b7725a', 'request_id': 'req-c35d4b39-7d5e-4af8-95fb-d0fa68bd652c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:48.482858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c35d4b39-7d5e-4af8-95fb-d0fa68bd652c admin admin] The token cache is being invalidate because project d97ee743a9034f3a9cf2c15db0b7725a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:48.483423 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c35d4b39-7d5e-4af8-95fb-d0fa68bd652c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:48.484326 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c35d4b39-7d5e-4af8-95fb-d0fa68bd652c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:48.486956 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 926/1865] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:48 2022] DELETE /identity/v3/projects/d97ee743a9034f3a9cf2c15db0b7725a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:48.488333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca0999-9c6e-4c83-a10d-58299c1bf24e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:48.488608 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca0999-9c6e-4c83-a10d-58299c1bf24e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:48.488884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca0999-9c6e-4c83-a10d-58299c1bf24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:48.512771 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-68ca0999-9c6e-4c83-a10d-58299c1bf24e None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:48.513974 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-68ca0999-9c6e-4c83-a10d-58299c1bf24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:48.527375 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae82a8-ed1d-4542-a36c-ade4950269fa admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:48.536980 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:48.587795 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c05ff-58ab-4b34-95f7-3cb04ff3ea09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:48.588540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c05ff-58ab-4b34-95f7-3cb04ff3ea09 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:48.589006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c05ff-58ab-4b34-95f7-3cb04ff3ea09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:48.589290 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c05ff-58ab-4b34-95f7-3cb04ff3ea09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:48.593236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5e7c05ff-58ab-4b34-95f7-3cb04ff3ea09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:48.601435 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 927/1866] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:48.602796 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b660d-d47c-42ca-8ac3-18dbaa4be8e4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:48.603354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b660d-d47c-42ca-8ac3-18dbaa4be8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:48.603489 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b660d-d47c-42ca-8ac3-18dbaa4be8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:48.662565 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-017b660d-d47c-42ca-8ac3-18dbaa4be8e4 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:48.667052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-017b660d-d47c-42ca-8ac3-18dbaa4be8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:48.668610 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c05ff-58ab-4b34-95f7-3cb04ff3ea09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:48.670117 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c05ff-58ab-4b34-95f7-3cb04ff3ea09 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:48.681685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 928/1867] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:48.683059 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e1097-ca95-4f23-a926-b2b523adfcfb None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:48.683985 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e1097-ca95-4f23-a926-b2b523adfcfb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:48.684350 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e1097-ca95-4f23-a926-b2b523adfcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:48.689308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 940/1868] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:48.690479 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c05ff-58ab-4b34-95f7-3cb04ff3ea09 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:48.692092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5e7c05ff-58ab-4b34-95f7-3cb04ff3ea09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:48.709744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c05ff-58ab-4b34-95f7-3cb04ff3ea09 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:48.711566 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e74e1097-ca95-4f23-a926-b2b523adfcfb None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:48.713176 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e74e1097-ca95-4f23-a926-b2b523adfcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:48.713493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dd709-2d18-4551-82df-d827feadab40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:48.714146 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dd709-2d18-4551-82df-d827feadab40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:48.715150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dd709-2d18-4551-82df-d827feadab40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:48.715440 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dd709-2d18-4551-82df-d827feadab40 admin admin] PATH_INFO: `/v3/users/a7f055ba26ea4b6db60108b6ce66514e` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:48.716787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37dd709-2d18-4551-82df-d827feadab40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7f055ba26ea4b6db60108b6ce66514e)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:48.718995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37dd709-2d18-4551-82df-d827feadab40 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:48.723260 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 929/1869] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:48.724434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac26b7-7d64-47fa-a145-be4497e3a215 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:48.724782 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac26b7-7d64-47fa-a145-be4497e3a215 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:48.725118 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac26b7-7d64-47fa-a145-be4497e3a215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:48.749325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-79ac26b7-7d64-47fa-a145-be4497e3a215 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:48.750233 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-79ac26b7-7d64-47fa-a145-be4497e3a215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:48.759346 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 930/1870] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:48.759975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac26b7-7d64-47fa-a145-be4497e3a215 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:48.773395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac26b7-7d64-47fa-a145-be4497e3a215 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:48.792110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41b740-18ce-429a-9d32-9bd7d4941c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:48.793117 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41b740-18ce-429a-9d32-9bd7d4941c82 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:48.793555 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41b740-18ce-429a-9d32-9bd7d4941c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:48.794203 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41b740-18ce-429a-9d32-9bd7d4941c82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:48.796029 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41b740-18ce-429a-9d32-9bd7d4941c82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:48.796344 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41b740-18ce-429a-9d32-9bd7d4941c82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:48.798538 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41b740-18ce-429a-9d32-9bd7d4941c82 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:48.813521 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 931/1871] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:48.813651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d37dd709-2d18-4551-82df-d827feadab40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7f055ba26ea4b6db60108b6ce66514e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:48.818270 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41b740-18ce-429a-9d32-9bd7d4941c82 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:48.832070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41b740-18ce-429a-9d32-9bd7d4941c82 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:48.835142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d37dd709-2d18-4551-82df-d827feadab40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7f055ba26ea4b6db60108b6ce66514e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:48.836811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d37dd709-2d18-4551-82df-d827feadab40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7f055ba26ea4b6db60108b6ce66514e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:48.839427 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d37dd709-2d18-4551-82df-d827feadab40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7f055ba26ea4b6db60108b6ce66514e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:48.842064 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4158a-9a5a-4343-82a8-dd00b0959e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:48.842966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4158a-9a5a-4343-82a8-dd00b0959e20 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:48.843403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4158a-9a5a-4343-82a8-dd00b0959e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:48.843905 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4158a-9a5a-4343-82a8-dd00b0959e20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:48.844806 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4158a-9a5a-4343-82a8-dd00b0959e20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:48.845317 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4158a-9a5a-4343-82a8-dd00b0959e20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:48.850243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4158a-9a5a-4343-82a8-dd00b0959e20 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:48.860809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d37dd709-2d18-4551-82df-d827feadab40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7f055ba26ea4b6db60108b6ce66514e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:48.872432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 932/1872] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:48.873629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4158a-9a5a-4343-82a8-dd00b0959e20 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:48.876070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7ac4158a-9a5a-4343-82a8-dd00b0959e20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:48.897643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4158a-9a5a-4343-82a8-dd00b0959e20 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:48.904005 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cb53e-3f6a-4951-b435-9883ec3d647f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:48.904558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 941/1873] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:48 2022] DELETE /identity/v3/users/a7f055ba26ea4b6db60108b6ce66514e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:48.904834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cb53e-3f6a-4951-b435-9883ec3d647f admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:48.905183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cb53e-3f6a-4951-b435-9883ec3d647f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:48.905523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cb53e-3f6a-4951-b435-9883ec3d647f admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:48.906430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cb53e-3f6a-4951-b435-9883ec3d647f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:48.907130 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dd709-2d18-4551-82df-d827feadab40 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:48.908706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d37dd709-2d18-4551-82df-d827feadab40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:48.908836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cb53e-3f6a-4951-b435-9883ec3d647f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:48.948343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 933/1874] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:48 2022] POST /identity/v3/projects => generated 381 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:48.950150 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de45a90-4c3e-423a-915b-7da9e20ad2e9 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:48.950422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de45a90-4c3e-423a-915b-7da9e20ad2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:48.950697 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de45a90-4c3e-423a-915b-7da9e20ad2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:48.964475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:48.971743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-5de45a90-4c3e-423a-915b-7da9e20ad2e9 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:48.973884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5de45a90-4c3e-423a-915b-7da9e20ad2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:48.987045 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:49.019786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dd709-2d18-4551-82df-d827feadab40 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:49.061160 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 934/1875] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:49.061970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de45a90-4c3e-423a-915b-7da9e20ad2e9 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:49.074433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de45a90-4c3e-423a-915b-7da9e20ad2e9 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:49.079720 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e929a78-dcd9-42e7-9d56-04fb1e770756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:49.080360 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e929a78-dcd9-42e7-9d56-04fb1e770756 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:49.080640 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e929a78-dcd9-42e7-9d56-04fb1e770756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:49.080918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e929a78-dcd9-42e7-9d56-04fb1e770756 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:49.081551 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e929a78-dcd9-42e7-9d56-04fb1e770756 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:49.083480 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e929a78-dcd9-42e7-9d56-04fb1e770756 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:49.105274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4620170-bf35-4518-9448-6fc5e17a01ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:49.106294 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4620170-bf35-4518-9448-6fc5e17a01ed admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:49.106582 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4620170-bf35-4518-9448-6fc5e17a01ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:49.106892 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4620170-bf35-4518-9448-6fc5e17a01ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:49.107665 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4620170-bf35-4518-9448-6fc5e17a01ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:49.109639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4620170-bf35-4518-9448-6fc5e17a01ed admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:49.115162 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 935/1876] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:49 2022] POST /identity/v3/users => generated 291 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:49.119284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e929a78-dcd9-42e7-9d56-04fb1e770756 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:49.121704 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9e929a78-dcd9-42e7-9d56-04fb1e770756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:49.144483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 942/1877] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:48 2022] POST /identity/v3/projects => generated 375 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:49.145682 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4620170-bf35-4518-9448-6fc5e17a01ed admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:49.257407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e929a78-dcd9-42e7-9d56-04fb1e770756 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:49.260748 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4620170-bf35-4518-9448-6fc5e17a01ed admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:49.265956 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a256a3-4de0-4c18-9633-4223eef8afba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:49.266580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a256a3-4de0-4c18-9633-4223eef8afba admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:49.266858 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a256a3-4de0-4c18-9633-4223eef8afba admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:49.267137 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a256a3-4de0-4c18-9633-4223eef8afba admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:49.267832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a256a3-4de0-4c18-9633-4223eef8afba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:49.269457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a256a3-4de0-4c18-9633-4223eef8afba admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:49.283632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 943/1878] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:49.283632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a256a3-4de0-4c18-9633-4223eef8afba admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:49.294047 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a256a3-4de0-4c18-9633-4223eef8afba admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:49.298675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd159a3-3ce5-4649-b814-6b6b6900679c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:49.299270 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd159a3-3ce5-4649-b814-6b6b6900679c admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:49.299582 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd159a3-3ce5-4649-b814-6b6b6900679c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:49.299861 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd159a3-3ce5-4649-b814-6b6b6900679c admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:49.300483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd159a3-3ce5-4649-b814-6b6b6900679c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:49.302381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd159a3-3ce5-4649-b814-6b6b6900679c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:49.312373 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23544bd2-3a27-4a6e-803a-4979b492cc15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:49.313001 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23544bd2-3a27-4a6e-803a-4979b492cc15 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:49.313440 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23544bd2-3a27-4a6e-803a-4979b492cc15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:49.313759 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23544bd2-3a27-4a6e-803a-4979b492cc15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:49.326499 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 944/1879] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:49 2022] POST /identity/v3/users => generated 288 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:49.326667 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23544bd2-3a27-4a6e-803a-4979b492cc15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:49.326850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd159a3-3ce5-4649-b814-6b6b6900679c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:49.326950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23544bd2-3a27-4a6e-803a-4979b492cc15 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:49.336060 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 936/1880] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:49.336737 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd159a3-3ce5-4649-b814-6b6b6900679c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:49.341924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041c2e7-c8f7-4799-b685-62bd06f57814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:49.342525 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041c2e7-c8f7-4799-b685-62bd06f57814 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:49.342807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041c2e7-c8f7-4799-b685-62bd06f57814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:49.343080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041c2e7-c8f7-4799-b685-62bd06f57814 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:49.344107 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041c2e7-c8f7-4799-b685-62bd06f57814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:49.344472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23544bd2-3a27-4a6e-803a-4979b492cc15 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:49.345386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041c2e7-c8f7-4799-b685-62bd06f57814 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:49.355809 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23544bd2-3a27-4a6e-803a-4979b492cc15 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:49.360924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 945/1881] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:49.362219 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041c2e7-c8f7-4799-b685-62bd06f57814 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:49.364044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daee7ad-14ed-4af1-96c7-2ca7f303365a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:49.364834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daee7ad-14ed-4af1-96c7-2ca7f303365a admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:49.364953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daee7ad-14ed-4af1-96c7-2ca7f303365a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:49.365196 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daee7ad-14ed-4af1-96c7-2ca7f303365a admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:49.365837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daee7ad-14ed-4af1-96c7-2ca7f303365a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:49.367703 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daee7ad-14ed-4af1-96c7-2ca7f303365a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:49.372694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041c2e7-c8f7-4799-b685-62bd06f57814 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:49.380425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 937/1882] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:49.381500 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1c1bc-cc45-45b8-beec-f96117cedf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:49.382728 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1c1bc-cc45-45b8-beec-f96117cedf3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:49.383155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1c1bc-cc45-45b8-beec-f96117cedf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:49.383872 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1c1bc-cc45-45b8-beec-f96117cedf3e admin admin] PATH_INFO: `/v3/projects/b0278d42d0c04365998dbd2d14f13ee8/users/dfb2e8a587184dba8fded45252b709b1/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:49.384499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daee7ad-14ed-4af1-96c7-2ca7f303365a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:49.398474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daee7ad-14ed-4af1-96c7-2ca7f303365a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:49.403586 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1c1bc-cc45-45b8-beec-f96117cedf3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0278d42d0c04365998dbd2d14f13ee8, user_id=dfb2e8a587184dba8fded45252b709b1, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:49.405707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab319db-c848-41f9-ac34-590e6224e062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:49.406329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab319db-c848-41f9-ac34-590e6224e062 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:49.407047 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab319db-c848-41f9-ac34-590e6224e062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:49.407381 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab319db-c848-41f9-ac34-590e6224e062 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:49.408094 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab319db-c848-41f9-ac34-590e6224e062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:49.410033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab319db-c848-41f9-ac34-590e6224e062 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:49.418722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1c1bc-cc45-45b8-beec-f96117cedf3e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:49.425231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 938/1883] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:49.426565 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab319db-c848-41f9-ac34-590e6224e062 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:49.439748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab319db-c848-41f9-ac34-590e6224e062 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:49.444764 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 946/1884] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:49 2022] PUT /identity/v3/projects/b0278d42d0c04365998dbd2d14f13ee8/users/dfb2e8a587184dba8fded45252b709b1/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:49.456800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66edbce-a38c-461e-b7d4-14fb325cf869 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:49.457274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66edbce-a38c-461e-b7d4-14fb325cf869 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:49.457746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66edbce-a38c-461e-b7d4-14fb325cf869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:49.482245 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:49.484150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:49.495579 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-a66edbce-a38c-461e-b7d4-14fb325cf869 None None] MFA Rules not processed for user `dfb2e8a587184dba8fded45252b709b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:49.497898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a66edbce-a38c-461e-b7d4-14fb325cf869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:49.518383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac03353-c645-4dac-9445-ee49d3d7d87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:49.519103 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac03353-c645-4dac-9445-ee49d3d7d87a admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:49.519487 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac03353-c645-4dac-9445-ee49d3d7d87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:49.519790 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac03353-c645-4dac-9445-ee49d3d7d87a admin admin] PATH_INFO: `/v3/projects/f4fedbcdda2c4b6dac6268c093008642/users/837394ab068e4f62aa1e725a59f083d0/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:49.551270 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac03353-c645-4dac-9445-ee49d3d7d87a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4fedbcdda2c4b6dac6268c093008642, user_id=837394ab068e4f62aa1e725a59f083d0, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:49.554517 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac03353-c645-4dac-9445-ee49d3d7d87a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:49.579965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 939/1885] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:49 2022] PUT /identity/v3/projects/f4fedbcdda2c4b6dac6268c093008642/users/837394ab068e4f62aa1e725a59f083d0/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:49.591099 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5bd50-a705-4316-ac16-282f8cb33adf None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:49.591380 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5bd50-a705-4316-ac16-282f8cb33adf None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:49.591800 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5bd50-a705-4316-ac16-282f8cb33adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:49.626950 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 947/1886] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:22:49.628417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66edbce-a38c-461e-b7d4-14fb325cf869 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:49.640566 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66edbce-a38c-461e-b7d4-14fb325cf869 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:49.650896 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-12b5bd50-a705-4316-ac16-282f8cb33adf None None] MFA Rules not processed for user `837394ab068e4f62aa1e725a59f083d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:49.652246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-12b5bd50-a705-4316-ac16-282f8cb33adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:49.729113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 940/1887] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:49.729797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5bd50-a705-4316-ac16-282f8cb33adf None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:49.732618 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-12b5bd50-a705-4316-ac16-282f8cb33adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:49.738951 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96551435-f211-4fcc-811d-3f9448e72d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:49.739800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96551435-f211-4fcc-811d-3f9448e72d28 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:49.740380 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96551435-f211-4fcc-811d-3f9448e72d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:49.740925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96551435-f211-4fcc-811d-3f9448e72d28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:49.741788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96551435-f211-4fcc-811d-3f9448e72d28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:49.744122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96551435-f211-4fcc-811d-3f9448e72d28 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:49.749352 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5bd50-a705-4316-ac16-282f8cb33adf None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:49.754220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2af87-c217-44d0-9f35-63313c034855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:49.754927 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2af87-c217-44d0-9f35-63313c034855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:49.755160 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2af87-c217-44d0-9f35-63313c034855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:49.755397 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2af87-c217-44d0-9f35-63313c034855 admin admin] PATH_INFO: `/v3/projects/cb99e62a65bc47f6804ccdc2534c244e` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:49.756561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2af87-c217-44d0-9f35-63313c034855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb99e62a65bc47f6804ccdc2534c244e)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:49.758444 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2af87-c217-44d0-9f35-63313c034855 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:49.780324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 948/1888] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:49 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:22:49.780324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96551435-f211-4fcc-811d-3f9448e72d28 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:49.785227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-96551435-f211-4fcc-811d-3f9448e72d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:49.817465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1fd2af87-c217-44d0-9f35-63313c034855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb99e62a65bc47f6804ccdc2534c244e', 'request_id': 'req-1fd2af87-c217-44d0-9f35-63313c034855'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:49.832126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1fd2af87-c217-44d0-9f35-63313c034855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb99e62a65bc47f6804ccdc2534c244e', 'request_id': 'req-1fd2af87-c217-44d0-9f35-63313c034855'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:49.842602 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1fd2af87-c217-44d0-9f35-63313c034855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb99e62a65bc47f6804ccdc2534c244e', 'request_id': 'req-1fd2af87-c217-44d0-9f35-63313c034855'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:49.858408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1fd2af87-c217-44d0-9f35-63313c034855 admin admin] The token cache is being invalidate because project cb99e62a65bc47f6804ccdc2534c244e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:49.858913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1fd2af87-c217-44d0-9f35-63313c034855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:49.859966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1fd2af87-c217-44d0-9f35-63313c034855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:49.862550 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 941/1889] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:49 2022] DELETE /identity/v3/projects/cb99e62a65bc47f6804ccdc2534c244e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:49.863672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2af87-c217-44d0-9f35-63313c034855 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:49.865798 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1fd2af87-c217-44d0-9f35-63313c034855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:49.892538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96551435-f211-4fcc-811d-3f9448e72d28 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:49.914889 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:49.939481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:49.979245 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f3ea7-67be-43c1-a40c-db5e728b4ee3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a889616f86346f48b7573969174ee57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:49.979909 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f3ea7-67be-43c1-a40c-db5e728b4ee3 service glance] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:49.980221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f3ea7-67be-43c1-a40c-db5e728b4ee3 service glance] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:49.980493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f3ea7-67be-43c1-a40c-db5e728b4ee3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:49.982530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-515f3ea7-67be-43c1-a40c-db5e728b4ee3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:49.991325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2af87-c217-44d0-9f35-63313c034855 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:50.009528 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9e88c-71c8-4b85-9528-bdacd793fab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:50.010146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9e88c-71c8-4b85-9528-bdacd793fab8 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:50.010432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9e88c-71c8-4b85-9528-bdacd793fab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:50.013407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9e88c-71c8-4b85-9528-bdacd793fab8 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:50.014121 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9e88c-71c8-4b85-9528-bdacd793fab8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:50.017130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9e88c-71c8-4b85-9528-bdacd793fab8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:50.058598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 942/1890] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:49 2022] POST /identity/v3/users => generated 298 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:50.058598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9e88c-71c8-4b85-9528-bdacd793fab8 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:50.058598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-98d9e88c-71c8-4b85-9528-bdacd793fab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:50.074908 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9e88c-71c8-4b85-9528-bdacd793fab8 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:50.075376 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f3ea7-67be-43c1-a40c-db5e728b4ee3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:50.078277 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f3ea7-67be-43c1-a40c-db5e728b4ee3 service glance] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:50.080812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720f5c4-3fda-4267-8638-1277c2820cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:50.081398 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720f5c4-3fda-4267-8638-1277c2820cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:50.081678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720f5c4-3fda-4267-8638-1277c2820cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:50.081953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720f5c4-3fda-4267-8638-1277c2820cc3 admin admin] PATH_INFO: `/v3/users/cc9878080b0745b488341af40d0e8e72` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:50.083247 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c720f5c4-3fda-4267-8638-1277c2820cc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc9878080b0745b488341af40d0e8e72)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:50.085206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c720f5c4-3fda-4267-8638-1277c2820cc3 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:50.091479 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 949/1891] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:50.093080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a81af7-bacc-4c1a-9c06-e89630f7dceb None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:50.093477 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a81af7-bacc-4c1a-9c06-e89630f7dceb None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:50.093858 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a81af7-bacc-4c1a-9c06-e89630f7dceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:50.119456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-19a81af7-bacc-4c1a-9c06-e89630f7dceb None None] MFA Rules not processed for user `2e71f9c1626f4f58916a5dd5c5987ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:50.121373 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-19a81af7-bacc-4c1a-9c06-e89630f7dceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:50.156159 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c720f5c4-3fda-4267-8638-1277c2820cc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc9878080b0745b488341af40d0e8e72'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:50.164586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c720f5c4-3fda-4267-8638-1277c2820cc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc9878080b0745b488341af40d0e8e72'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:50.165323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c720f5c4-3fda-4267-8638-1277c2820cc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc9878080b0745b488341af40d0e8e72'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:50.166607 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c720f5c4-3fda-4267-8638-1277c2820cc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc9878080b0745b488341af40d0e8e72'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:50.184213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c720f5c4-3fda-4267-8638-1277c2820cc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc9878080b0745b488341af40d0e8e72'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:50.190937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 950/1892] 10.222.0.23 () {58 vars in 1015 bytes} [Thu Oct 6 12:22:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:50.191836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a81af7-bacc-4c1a-9c06-e89630f7dceb None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:50.193450 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-19a81af7-bacc-4c1a-9c06-e89630f7dceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:50.208807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a81af7-bacc-4c1a-9c06-e89630f7dceb None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:50.210570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 943/1893] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:50 2022] DELETE /identity/v3/users/cc9878080b0745b488341af40d0e8e72 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:50.307695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5199246-1817-41a3-b67d-e86b3bf817ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:50.308602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5199246-1817-41a3-b67d-e86b3bf817ec admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:50.309126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5199246-1817-41a3-b67d-e86b3bf817ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:50.309495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5199246-1817-41a3-b67d-e86b3bf817ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:50.310383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5199246-1817-41a3-b67d-e86b3bf817ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:50.312571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5199246-1817-41a3-b67d-e86b3bf817ec admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:50.328650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 951/1894] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:50.332632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720f5c4-3fda-4267-8638-1277c2820cc3 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:50.364478 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720f5c4-3fda-4267-8638-1277c2820cc3 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:50.376944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a1003-668b-4c5b-b7a0-02b05cd4f224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:50.377578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a1003-668b-4c5b-b7a0-02b05cd4f224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:50.377868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a1003-668b-4c5b-b7a0-02b05cd4f224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:50.378148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a1003-668b-4c5b-b7a0-02b05cd4f224 admin admin] PATH_INFO: `/v3/projects/d74251e7a24545c3bc82fe30518ffe57/users/7202bddd8c2647a59ec3d86d08f40069/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:50.414922 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a1003-668b-4c5b-b7a0-02b05cd4f224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d74251e7a24545c3bc82fe30518ffe57, user_id=7202bddd8c2647a59ec3d86d08f40069, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:50.418147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a1003-668b-4c5b-b7a0-02b05cd4f224 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:50.467105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5199246-1817-41a3-b67d-e86b3bf817ec admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:50.471183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f5199246-1817-41a3-b67d-e86b3bf817ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:50.484867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 944/1895] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:50 2022] PUT /identity/v3/projects/d74251e7a24545c3bc82fe30518ffe57/users/7202bddd8c2647a59ec3d86d08f40069/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:50.486215 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a1003-668b-4c5b-b7a0-02b05cd4f224 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:50.521548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:50.629246 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5199246-1817-41a3-b67d-e86b3bf817ec admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:50.658030 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a1003-668b-4c5b-b7a0-02b05cd4f224 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:50.667209 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa2369-1962-4764-9922-40d2b6ef951c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:50.668199 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa2369-1962-4764-9922-40d2b6ef951c admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:50.668698 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa2369-1962-4764-9922-40d2b6ef951c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:50.669077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa2369-1962-4764-9922-40d2b6ef951c admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:50.669832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa2369-1962-4764-9922-40d2b6ef951c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:50.671719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa2369-1962-4764-9922-40d2b6ef951c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:50.687680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 945/1896] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:50.689116 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa2369-1962-4764-9922-40d2b6ef951c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:50.702560 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:50.734800 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa2369-1962-4764-9922-40d2b6ef951c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:50.747452 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92af41-f928-4707-8318-349ecffc433c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:50.748611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92af41-f928-4707-8318-349ecffc433c service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:50.749151 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92af41-f928-4707-8318-349ecffc433c service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:50.749645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92af41-f928-4707-8318-349ecffc433c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:50.753089 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753f24e-30ee-4a79-b886-3dc15b0fb3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:50.754838 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753f24e-30ee-4a79-b886-3dc15b0fb3cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:50.755848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753f24e-30ee-4a79-b886-3dc15b0fb3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:50.756137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753f24e-30ee-4a79-b886-3dc15b0fb3cf admin admin] PATH_INFO: `/v3/projects/d74251e7a24545c3bc82fe30518ffe57/users/7202bddd8c2647a59ec3d86d08f40069/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:50.758794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753f24e-30ee-4a79-b886-3dc15b0fb3cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d74251e7a24545c3bc82fe30518ffe57, user_id=7202bddd8c2647a59ec3d86d08f40069, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:50.761980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753f24e-30ee-4a79-b886-3dc15b0fb3cf admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:50.772123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92af41-f928-4707-8318-349ecffc433c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:50.775477 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92af41-f928-4707-8318-349ecffc433c service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:50.799954 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 946/1898] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:50 2022] PUT /identity/v3/projects/d74251e7a24545c3bc82fe30518ffe57/users/7202bddd8c2647a59ec3d86d08f40069/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:50.801133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 952/1898] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:50.801399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6980b-efb8-40e1-bff5-4fc5905ca175 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:50.801692 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6980b-efb8-40e1-bff5-4fc5905ca175 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:50.801964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6980b-efb8-40e1-bff5-4fc5905ca175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:50.897960 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c8b6980b-efb8-40e1-bff5-4fc5905ca175 None None] MFA Rules not processed for user `7202bddd8c2647a59ec3d86d08f40069`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:50.899268 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c8b6980b-efb8-40e1-bff5-4fc5905ca175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:51.095349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 947/1899] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:51.107728 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92af41-f928-4707-8318-349ecffc433c service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:51.109464 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1b92af41-f928-4707-8318-349ecffc433c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:51.263090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92af41-f928-4707-8318-349ecffc433c service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:51.343031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c9c7b-64d7-4369-9afc-d2b2f36a0c3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:51.344033 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c9c7b-64d7-4369-9afc-d2b2f36a0c3d service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:51.344500 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c9c7b-64d7-4369-9afc-d2b2f36a0c3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:51.345025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c9c7b-64d7-4369-9afc-d2b2f36a0c3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:51.359888 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c9c7b-64d7-4369-9afc-d2b2f36a0c3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:51.363098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c9c7b-64d7-4369-9afc-d2b2f36a0c3d service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:51.374830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 953/1900] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:51.397193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6980b-efb8-40e1-bff5-4fc5905ca175 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:51.400603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c8b6980b-efb8-40e1-bff5-4fc5905ca175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:51.561116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c9c7b-64d7-4369-9afc-d2b2f36a0c3d service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:51.585344 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c9c7b-64d7-4369-9afc-d2b2f36a0c3d service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:51.597203 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb36d74-799f-4d0e-a3ea-8604c91e1839 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:51.597753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb36d74-799f-4d0e-a3ea-8604c91e1839 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:51.598033 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb36d74-799f-4d0e-a3ea-8604c91e1839 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:51.602464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6980b-efb8-40e1-bff5-4fc5905ca175 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:51.605229 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb36d74-799f-4d0e-a3ea-8604c91e1839 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:51.608035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b612bea7-7ec2-4d6a-9ece-440b367b6be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:51.608897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612bea7-7ec2-4d6a-9ece-440b367b6be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:51.609283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612bea7-7ec2-4d6a-9ece-440b367b6be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:51.609668 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612bea7-7ec2-4d6a-9ece-440b367b6be9 admin admin] PATH_INFO: `/v3/projects/6ec7db3bc6fd4cd494124b316a11b31f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:51.611014 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b612bea7-7ec2-4d6a-9ece-440b367b6be9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ec7db3bc6fd4cd494124b316a11b31f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:51.617977 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cbb36d74-799f-4d0e-a3ea-8604c91e1839 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:51.619827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b612bea7-7ec2-4d6a-9ece-440b367b6be9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:51.765488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b612bea7-7ec2-4d6a-9ece-440b367b6be9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ec7db3bc6fd4cd494124b316a11b31f', 'request_id': 'req-b612bea7-7ec2-4d6a-9ece-440b367b6be9'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:51.786135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b612bea7-7ec2-4d6a-9ece-440b367b6be9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ec7db3bc6fd4cd494124b316a11b31f', 'request_id': 'req-b612bea7-7ec2-4d6a-9ece-440b367b6be9'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:51.791861 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:51.816669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b612bea7-7ec2-4d6a-9ece-440b367b6be9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ec7db3bc6fd4cd494124b316a11b31f', 'request_id': 'req-b612bea7-7ec2-4d6a-9ece-440b367b6be9'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:51.881591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b612bea7-7ec2-4d6a-9ece-440b367b6be9 admin admin] The token cache is being invalidate because project 6ec7db3bc6fd4cd494124b316a11b31f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:51.882224 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b612bea7-7ec2-4d6a-9ece-440b367b6be9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:51.883130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b612bea7-7ec2-4d6a-9ece-440b367b6be9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:51.887196 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 948/1901] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:51 2022] DELETE /identity/v3/projects/6ec7db3bc6fd4cd494124b316a11b31f => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:51.893652 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eabbcc-751c-4fcb-8a05-8bf6a27c1006 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:51.894145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eabbcc-751c-4fcb-8a05-8bf6a27c1006 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:51.894869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eabbcc-751c-4fcb-8a05-8bf6a27c1006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:51.896978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb36d74-799f-4d0e-a3ea-8604c91e1839 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:51.898519 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb36d74-799f-4d0e-a3ea-8604c91e1839 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:51.919291 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:51.958645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 954/1902] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:51.964890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-b1eabbcc-751c-4fcb-8a05-8bf6a27c1006 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:51.970866 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b1eabbcc-751c-4fcb-8a05-8bf6a27c1006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:52.192775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 949/1903] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:52.202726 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b0899-d8dd-4375-a10f-5c4f2a5555bd None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:52.203261 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b0899-d8dd-4375-a10f-5c4f2a5555bd None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:52.203466 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b0899-d8dd-4375-a10f-5c4f2a5555bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:52.250092 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ef7b0899-d8dd-4375-a10f-5c4f2a5555bd None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:52.255116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ef7b0899-d8dd-4375-a10f-5c4f2a5555bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:52.280518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 955/1904] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:22:52.280739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eabbcc-751c-4fcb-8a05-8bf6a27c1006 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:52.296522 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eabbcc-751c-4fcb-8a05-8bf6a27c1006 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:52.313774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6de40-2bf8-4006-8048-0c9bf34fe4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:52.314637 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6de40-2bf8-4006-8048-0c9bf34fe4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:52.315047 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6de40-2bf8-4006-8048-0c9bf34fe4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:52.315445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6de40-2bf8-4006-8048-0c9bf34fe4e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:52.316562 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6de40-2bf8-4006-8048-0c9bf34fe4e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:52.316998 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6de40-2bf8-4006-8048-0c9bf34fe4e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:52.319895 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6de40-2bf8-4006-8048-0c9bf34fe4e1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:52.372848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 950/1905] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:22:52.377035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b0899-d8dd-4375-a10f-5c4f2a5555bd None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:52.390199 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b0899-d8dd-4375-a10f-5c4f2a5555bd None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:52.408677 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5774a-8f14-41e1-ba2a-afa60aacfc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:52.409111 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5774a-8f14-41e1-ba2a-afa60aacfc78 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:52.409399 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5774a-8f14-41e1-ba2a-afa60aacfc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:52.409975 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5774a-8f14-41e1-ba2a-afa60aacfc78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:52.410689 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5774a-8f14-41e1-ba2a-afa60aacfc78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:52.413057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5774a-8f14-41e1-ba2a-afa60aacfc78 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:52.479793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 956/1906] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:52 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:52.484307 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6de40-2bf8-4006-8048-0c9bf34fe4e1 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:52.523246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6de40-2bf8-4006-8048-0c9bf34fe4e1 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:52.660263 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5527f20-c679-4cd6-b82a-f2ba8b92010c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:52.662393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5527f20-c679-4cd6-b82a-f2ba8b92010c admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:52.662828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5527f20-c679-4cd6-b82a-f2ba8b92010c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:52.663216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5527f20-c679-4cd6-b82a-f2ba8b92010c admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:52.664016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5527f20-c679-4cd6-b82a-f2ba8b92010c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:52.666220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5527f20-c679-4cd6-b82a-f2ba8b92010c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:52.725893 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 951/1907] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:52 2022] POST /identity/v3/users => generated 292 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:52.729863 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5774a-8f14-41e1-ba2a-afa60aacfc78 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:52.741947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5774a-8f14-41e1-ba2a-afa60aacfc78 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:52.747268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a4dff-6a06-4dac-adb3-6477ccf22642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:52.747921 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a4dff-6a06-4dac-adb3-6477ccf22642 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:52.748207 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a4dff-6a06-4dac-adb3-6477ccf22642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:52.748473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a4dff-6a06-4dac-adb3-6477ccf22642 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:52.749123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a4dff-6a06-4dac-adb3-6477ccf22642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:52.750725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a4dff-6a06-4dac-adb3-6477ccf22642 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:52.772919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5527f20-c679-4cd6-b82a-f2ba8b92010c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:52.775105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 957/1908] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:52.796071 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5527f20-c679-4cd6-b82a-f2ba8b92010c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:52.804944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0ff7c-ce6f-4b05-89b9-05fb8595351c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:52.805938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0ff7c-ce6f-4b05-89b9-05fb8595351c admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:52.806353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0ff7c-ce6f-4b05-89b9-05fb8595351c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:52.806643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0ff7c-ce6f-4b05-89b9-05fb8595351c admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:52.807309 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0ff7c-ce6f-4b05-89b9-05fb8595351c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:52.809085 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0ff7c-ce6f-4b05-89b9-05fb8595351c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:52.827428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 952/1909] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:22:52.833451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a4dff-6a06-4dac-adb3-6477ccf22642 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:52.848466 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a4dff-6a06-4dac-adb3-6477ccf22642 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:52.856943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81095dc-3f5f-4cf0-b4a3-bd3ac3086479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:52.857730 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81095dc-3f5f-4cf0-b4a3-bd3ac3086479 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:52.858093 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81095dc-3f5f-4cf0-b4a3-bd3ac3086479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:52.858444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81095dc-3f5f-4cf0-b4a3-bd3ac3086479 admin admin] PATH_INFO: `/v3/projects/dd1c2d95526e4e8aba8fb7fd44eed029/users/c9695c5e5f7f4889a5d12bdca17819e5/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:52.877813 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81095dc-3f5f-4cf0-b4a3-bd3ac3086479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd1c2d95526e4e8aba8fb7fd44eed029, user_id=c9695c5e5f7f4889a5d12bdca17819e5, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:52.880622 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81095dc-3f5f-4cf0-b4a3-bd3ac3086479 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:52.901364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 958/1910] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:52 2022] PUT /identity/v3/projects/dd1c2d95526e4e8aba8fb7fd44eed029/users/c9695c5e5f7f4889a5d12bdca17819e5/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:52.910618 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896e6b3-1518-4c4c-b5a5-fe7001c46257 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:52.910916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896e6b3-1518-4c4c-b5a5-fe7001c46257 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:52.911174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896e6b3-1518-4c4c-b5a5-fe7001c46257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:52.944016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f896e6b3-1518-4c4c-b5a5-fe7001c46257 None None] MFA Rules not processed for user `c9695c5e5f7f4889a5d12bdca17819e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:52.945262 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f896e6b3-1518-4c4c-b5a5-fe7001c46257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:53.013491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:53.019606 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 953/1911] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:53.024403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81095dc-3f5f-4cf0-b4a3-bd3ac3086479 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:53.095526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81095dc-3f5f-4cf0-b4a3-bd3ac3086479 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:53.100215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2708e4-c9a5-4f1c-9acf-227a2884aae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:53.100827 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2708e4-c9a5-4f1c-9acf-227a2884aae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:53.101107 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2708e4-c9a5-4f1c-9acf-227a2884aae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:53.101398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2708e4-c9a5-4f1c-9acf-227a2884aae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:53.102043 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2708e4-c9a5-4f1c-9acf-227a2884aae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:53.104065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2708e4-c9a5-4f1c-9acf-227a2884aae4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:53.132685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 959/1912] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:53 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:53.137178 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896e6b3-1518-4c4c-b5a5-fe7001c46257 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:53.150355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896e6b3-1518-4c4c-b5a5-fe7001c46257 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:53.213644 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89b7a9-0527-4b7b-b030-be5acaa8b82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:53.220589 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89b7a9-0527-4b7b-b030-be5acaa8b82f admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:53.220915 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89b7a9-0527-4b7b-b030-be5acaa8b82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:53.221202 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89b7a9-0527-4b7b-b030-be5acaa8b82f admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:53.221892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89b7a9-0527-4b7b-b030-be5acaa8b82f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:53.223891 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89b7a9-0527-4b7b-b030-be5acaa8b82f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:53.265734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 954/1913] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:53 2022] POST /identity/v3/users => generated 298 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:53.267452 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2708e4-c9a5-4f1c-9acf-227a2884aae4 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:53.280655 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2708e4-c9a5-4f1c-9acf-227a2884aae4 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:53.296890 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ba8c6-888b-49a5-926e-d1eef335400c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:53.297813 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ba8c6-888b-49a5-926e-d1eef335400c admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:53.298260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ba8c6-888b-49a5-926e-d1eef335400c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:53.298692 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ba8c6-888b-49a5-926e-d1eef335400c admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:53.299478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ba8c6-888b-49a5-926e-d1eef335400c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:53.301634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ba8c6-888b-49a5-926e-d1eef335400c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:53.324232 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 960/1914] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:53.324403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89b7a9-0527-4b7b-b030-be5acaa8b82f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:53.335270 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89b7a9-0527-4b7b-b030-be5acaa8b82f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:53.341739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22a572-d5bd-4ffd-b928-12d167037803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:53.342553 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22a572-d5bd-4ffd-b928-12d167037803 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:53.342955 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22a572-d5bd-4ffd-b928-12d167037803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:53.344181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22a572-d5bd-4ffd-b928-12d167037803 admin admin] PATH_INFO: `/v3/projects/f2301ca0dee846ca818a6a2ab98764c0/users/51a749bdaa444ddcadd9d265856ccd4f/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:53.376162 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22a572-d5bd-4ffd-b928-12d167037803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2301ca0dee846ca818a6a2ab98764c0, user_id=51a749bdaa444ddcadd9d265856ccd4f, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:53.379032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22a572-d5bd-4ffd-b928-12d167037803 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:53.415656 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 955/1915] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:53 2022] PUT /identity/v3/projects/f2301ca0dee846ca818a6a2ab98764c0/users/51a749bdaa444ddcadd9d265856ccd4f/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:53.417925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ba8c6-888b-49a5-926e-d1eef335400c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:53.432606 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ba8c6-888b-49a5-926e-d1eef335400c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:53.512098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:53.529878 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819154c-0058-46e2-b88b-6cae499e4847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:53.530943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819154c-0058-46e2-b88b-6cae499e4847 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:53.531386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819154c-0058-46e2-b88b-6cae499e4847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:53.533588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819154c-0058-46e2-b88b-6cae499e4847 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:53.534222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819154c-0058-46e2-b88b-6cae499e4847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:53.539440 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819154c-0058-46e2-b88b-6cae499e4847 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:53.573847 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 961/1916] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:53.575045 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22a572-d5bd-4ffd-b928-12d167037803 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:53.587213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22a572-d5bd-4ffd-b928-12d167037803 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:53.613827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12b41d-3bfd-4da2-8803-290ba919be11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:53.615960 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12b41d-3bfd-4da2-8803-290ba919be11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:53.616452 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12b41d-3bfd-4da2-8803-290ba919be11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:53.616985 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12b41d-3bfd-4da2-8803-290ba919be11 admin admin] PATH_INFO: `/v3/projects/f2301ca0dee846ca818a6a2ab98764c0/users/51a749bdaa444ddcadd9d265856ccd4f/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:53.624192 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12b41d-3bfd-4da2-8803-290ba919be11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2301ca0dee846ca818a6a2ab98764c0, user_id=51a749bdaa444ddcadd9d265856ccd4f, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:53.624953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa0f1d-e27e-4217-bd1e-049932626ea3 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:53.625245 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa0f1d-e27e-4217-bd1e-049932626ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:53.625530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa0f1d-e27e-4217-bd1e-049932626ea3 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:53.626881 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 962/1917] 10.222.0.33 () {54 vars in 921 bytes} [Thu Oct 6 12:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:22:53.631643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12b41d-3bfd-4da2-8803-290ba919be11 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:53.667870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 956/1918] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:53 2022] PUT /identity/v3/projects/f2301ca0dee846ca818a6a2ab98764c0/users/51a749bdaa444ddcadd9d265856ccd4f/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:53.671757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96440b11-5933-4832-bf10-b6cee894c0ad None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:53.672067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96440b11-5933-4832-bf10-b6cee894c0ad None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:53.672348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96440b11-5933-4832-bf10-b6cee894c0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:53.755222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-96440b11-5933-4832-bf10-b6cee894c0ad None None] MFA Rules not processed for user `51a749bdaa444ddcadd9d265856ccd4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:53.757441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-96440b11-5933-4832-bf10-b6cee894c0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:53.797239 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42fc60-5471-455b-8088-156177ad058f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:53.797746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42fc60-5471-455b-8088-156177ad058f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:53.797919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42fc60-5471-455b-8088-156177ad058f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:53.835931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-0f42fc60-5471-455b-8088-156177ad058f None None] MFA Rules not processed for user `5a46e63d34564d34b6b5752706375798`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:53.838719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0f42fc60-5471-455b-8088-156177ad058f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:53.945047 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 963/1919] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:53.958004 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96440b11-5933-4832-bf10-b6cee894c0ad None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:53.961343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-96440b11-5933-4832-bf10-b6cee894c0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:53.969344 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 957/1920] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Oct 6 12:22:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:22:54.086481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96440b11-5933-4832-bf10-b6cee894c0ad None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:54.090216 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d355cf4-36c3-4b82-89bb-03af0b201f5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:54.091475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d355cf4-36c3-4b82-89bb-03af0b201f5b service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:54.091475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d355cf4-36c3-4b82-89bb-03af0b201f5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:54.091815 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d355cf4-36c3-4b82-89bb-03af0b201f5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:54.111638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d355cf4-36c3-4b82-89bb-03af0b201f5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:54.113688 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d355cf4-36c3-4b82-89bb-03af0b201f5b service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:54.126198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 964/1921] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:54.187828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18c2f0-0859-490f-9a26-9bd8127e2d0c None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:54.188332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18c2f0-0859-490f-9a26-9bd8127e2d0c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:54.189407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18c2f0-0859-490f-9a26-9bd8127e2d0c None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:54.195357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 958/1922] 10.222.0.33 () {54 vars in 921 bytes} [Thu Oct 6 12:22:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:22:54.235799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06dec4-005f-4208-82ca-a02d06937be6 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:54.235799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06dec4-005f-4208-82ca-a02d06937be6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:54.236189 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06dec4-005f-4208-82ca-a02d06937be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:54.250031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18c2f0-0859-490f-9a26-9bd8127e2d0c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:54.270617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18c2f0-0859-490f-9a26-9bd8127e2d0c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:54.279487 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a9699-64a9-4065-9252-2f1e99260552 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:54.280146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a9699-64a9-4065-9252-2f1e99260552 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:54.280431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a9699-64a9-4065-9252-2f1e99260552 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:54.280709 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a9699-64a9-4065-9252-2f1e99260552 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:54.293300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-1f06dec4-005f-4208-82ca-a02d06937be6 None None] MFA Rules not processed for user `5a46e63d34564d34b6b5752706375798`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:54.296933 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1f06dec4-005f-4208-82ca-a02d06937be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:54.331328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 965/1923] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Oct 6 12:22:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:54.359355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:54.385636 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a9699-64a9-4065-9252-2f1e99260552 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:54.387742 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a9699-64a9-4065-9252-2f1e99260552 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:54.400300 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 959/1924] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:54.445922 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06dec4-005f-4208-82ca-a02d06937be6 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:54.452425 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1f06dec4-005f-4208-82ca-a02d06937be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:54.479609 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06dec4-005f-4208-82ca-a02d06937be6 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:54.500113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b006da-d36c-4ed4-b5df-ca23242aecc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:54.500811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b006da-d36c-4ed4-b5df-ca23242aecc2 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:54.501117 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b006da-d36c-4ed4-b5df-ca23242aecc2 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:54.501401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b006da-d36c-4ed4-b5df-ca23242aecc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:54.516185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a9699-64a9-4065-9252-2f1e99260552 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:54.516360 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b006da-d36c-4ed4-b5df-ca23242aecc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:54.517618 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bd6a9699-64a9-4065-9252-2f1e99260552 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:54.518153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b006da-d36c-4ed4-b5df-ca23242aecc2 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:54.552604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 966/1925] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:54.666902 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a9699-64a9-4065-9252-2f1e99260552 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:54.672853 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41ffee-93e3-412a-bb3f-36c41861d543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:54.673761 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41ffee-93e3-412a-bb3f-36c41861d543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:54.674224 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41ffee-93e3-412a-bb3f-36c41861d543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:54.674682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41ffee-93e3-412a-bb3f-36c41861d543 admin admin] PATH_INFO: `/v3/users/4e5fd1231b034e8798ea0071e0a71f21` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:54.675911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41ffee-93e3-412a-bb3f-36c41861d543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e5fd1231b034e8798ea0071e0a71f21)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:54.677846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41ffee-93e3-412a-bb3f-36c41861d543 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:54.680837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b006da-d36c-4ed4-b5df-ca23242aecc2 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:54.700759 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b006da-d36c-4ed4-b5df-ca23242aecc2 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:54.718326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c365b-02c2-4c39-9336-2cc64c4565e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:54.718326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c365b-02c2-4c39-9336-2cc64c4565e4 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:54.718326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c365b-02c2-4c39-9336-2cc64c4565e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:54.718326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c365b-02c2-4c39-9336-2cc64c4565e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:54.749863 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6c41ffee-93e3-412a-bb3f-36c41861d543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e5fd1231b034e8798ea0071e0a71f21'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:54.755936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c365b-02c2-4c39-9336-2cc64c4565e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:54.760054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c365b-02c2-4c39-9336-2cc64c4565e4 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:54.762303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6c41ffee-93e3-412a-bb3f-36c41861d543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e5fd1231b034e8798ea0071e0a71f21'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:54.763078 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6c41ffee-93e3-412a-bb3f-36c41861d543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e5fd1231b034e8798ea0071e0a71f21'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:54.763794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6c41ffee-93e3-412a-bb3f-36c41861d543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e5fd1231b034e8798ea0071e0a71f21'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:54.777424 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-6c41ffee-93e3-412a-bb3f-36c41861d543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e5fd1231b034e8798ea0071e0a71f21'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:54.783881 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 967/1926] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:22:54.821781 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 960/1927] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:54 2022] DELETE /identity/v3/users/4e5fd1231b034e8798ea0071e0a71f21 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:54.825356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a0f40-a714-4dea-9cc4-ab414735e627 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:54.825881 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a0f40-a714-4dea-9cc4-ab414735e627 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:54.826485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a0f40-a714-4dea-9cc4-ab414735e627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:54.857454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-273a0f40-a714-4dea-9cc4-ab414735e627 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:54.861371 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-273a0f40-a714-4dea-9cc4-ab414735e627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:54.921706 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41ffee-93e3-412a-bb3f-36c41861d543 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:54.924214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6c41ffee-93e3-412a-bb3f-36c41861d543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:54.976940 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:55.011931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 968/1928] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:55.059342 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41ffee-93e3-412a-bb3f-36c41861d543 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:55.067213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a0f40-a714-4dea-9cc4-ab414735e627 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:55.069201 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-273a0f40-a714-4dea-9cc4-ab414735e627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:55.076829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1a414-a31d-4c35-8d62-a69aa2d2efa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:55.077576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1a414-a31d-4c35-8d62-a69aa2d2efa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:55.077848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1a414-a31d-4c35-8d62-a69aa2d2efa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:55.078174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1a414-a31d-4c35-8d62-a69aa2d2efa5 admin admin] PATH_INFO: `/v3/users/c9695c5e5f7f4889a5d12bdca17819e5` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:55.079770 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1a414-a31d-4c35-8d62-a69aa2d2efa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9695c5e5f7f4889a5d12bdca17819e5)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:55.082004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1a414-a31d-4c35-8d62-a69aa2d2efa5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:55.094519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:55.203982 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a0f40-a714-4dea-9cc4-ab414735e627 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:55.211397 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2305eb0-d840-4955-b229-22d26bb9a3c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:55.212201 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2305eb0-d840-4955-b229-22d26bb9a3c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:55.212542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2305eb0-d840-4955-b229-22d26bb9a3c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:55.212824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2305eb0-d840-4955-b229-22d26bb9a3c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:55.214588 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b6d1a414-a31d-4c35-8d62-a69aa2d2efa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9695c5e5f7f4889a5d12bdca17819e5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:55.230823 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b6d1a414-a31d-4c35-8d62-a69aa2d2efa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9695c5e5f7f4889a5d12bdca17819e5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:55.232775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b6d1a414-a31d-4c35-8d62-a69aa2d2efa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9695c5e5f7f4889a5d12bdca17819e5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:55.233587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b6d1a414-a31d-4c35-8d62-a69aa2d2efa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9695c5e5f7f4889a5d12bdca17819e5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:55.235720 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2305eb0-d840-4955-b229-22d26bb9a3c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:55.237811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2305eb0-d840-4955-b229-22d26bb9a3c2 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:55.250061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b6d1a414-a31d-4c35-8d62-a69aa2d2efa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9695c5e5f7f4889a5d12bdca17819e5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:55.258059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 969/1929] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:55.259460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2305eb0-d840-4955-b229-22d26bb9a3c2 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:55.261150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a2305eb0-d840-4955-b229-22d26bb9a3c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:55.283857 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2305eb0-d840-4955-b229-22d26bb9a3c2 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:55.298002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bec855-5db5-4fb8-b005-0e253e9930ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:55.299063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bec855-5db5-4fb8-b005-0e253e9930ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:55.299349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bec855-5db5-4fb8-b005-0e253e9930ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:55.300443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bec855-5db5-4fb8-b005-0e253e9930ed admin admin] PATH_INFO: `/v3/users/dfb2e8a587184dba8fded45252b709b1` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:55.301952 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bec855-5db5-4fb8-b005-0e253e9930ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfb2e8a587184dba8fded45252b709b1)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:55.304670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bec855-5db5-4fb8-b005-0e253e9930ed admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:55.311009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 961/1930] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:54 2022] DELETE /identity/v3/users/c9695c5e5f7f4889a5d12bdca17819e5 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:55.322273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6980433-ae67-4393-9935-84bae079d476 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:55.322769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6980433-ae67-4393-9935-84bae079d476 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:55.323347 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6980433-ae67-4393-9935-84bae079d476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:55.361986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-b6980433-ae67-4393-9935-84bae079d476 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:55.363398 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b6980433-ae67-4393-9935-84bae079d476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:55.402824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-18bec855-5db5-4fb8-b005-0e253e9930ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfb2e8a587184dba8fded45252b709b1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:55.413227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-18bec855-5db5-4fb8-b005-0e253e9930ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfb2e8a587184dba8fded45252b709b1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:55.414285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-18bec855-5db5-4fb8-b005-0e253e9930ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfb2e8a587184dba8fded45252b709b1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:55.415776 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-18bec855-5db5-4fb8-b005-0e253e9930ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfb2e8a587184dba8fded45252b709b1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:55.432272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-18bec855-5db5-4fb8-b005-0e253e9930ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfb2e8a587184dba8fded45252b709b1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:55.465763 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 970/1931] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:55 2022] DELETE /identity/v3/users/dfb2e8a587184dba8fded45252b709b1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:55.470499 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f4192-d27c-4118-ba6a-f569ebe0def8 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:55.470788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f4192-d27c-4118-ba6a-f569ebe0def8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:55.471234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f4192-d27c-4118-ba6a-f569ebe0def8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:55.495312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 962/1932] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:22:55.496996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ee7f4192-d27c-4118-ba6a-f569ebe0def8 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:55.498366 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ee7f4192-d27c-4118-ba6a-f569ebe0def8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:55.501952 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6980433-ae67-4393-9935-84bae079d476 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:55.504567 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b6980433-ae67-4393-9935-84bae079d476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:55.570029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:55.598443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 971/1933] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:55.598659 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6980433-ae67-4393-9935-84bae079d476 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:55.607131 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f4192-d27c-4118-ba6a-f569ebe0def8 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:55.649852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:55.707575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb90e0-d269-4c3a-baa6-9755767c46f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:55.707575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb90e0-d269-4c3a-baa6-9755767c46f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:55.707575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb90e0-d269-4c3a-baa6-9755767c46f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:55.707575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb90e0-d269-4c3a-baa6-9755767c46f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:55.718774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f4192-d27c-4118-ba6a-f569ebe0def8 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:55.721608 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb90e0-d269-4c3a-baa6-9755767c46f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:55.723346 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb90e0-d269-4c3a-baa6-9755767c46f8 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:55.723733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0ddff-ceae-4e76-b530-d70bad0302d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:55.724423 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0ddff-ceae-4e76-b530-d70bad0302d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:55.724812 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0ddff-ceae-4e76-b530-d70bad0302d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:55.725180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0ddff-ceae-4e76-b530-d70bad0302d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:55.734276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 963/1934] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:22:55.736931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0ddff-ceae-4e76-b530-d70bad0302d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:55.738576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0ddff-ceae-4e76-b530-d70bad0302d0 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:55.751824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 972/1935] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:22:55.801522 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb90e0-d269-4c3a-baa6-9755767c46f8 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:55.804125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b0fb90e0-d269-4c3a-baa6-9755767c46f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:55.821833 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb90e0-d269-4c3a-baa6-9755767c46f8 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:55.827762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61667921-156b-4ca0-b64c-73f5e5eddc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:55.828913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61667921-156b-4ca0-b64c-73f5e5eddc14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:55.829200 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61667921-156b-4ca0-b64c-73f5e5eddc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:55.829480 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61667921-156b-4ca0-b64c-73f5e5eddc14 admin admin] PATH_INFO: `/v3/projects/45d44dd7707a45399e078892328d5e85` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:55.831137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61667921-156b-4ca0-b64c-73f5e5eddc14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d44dd7707a45399e078892328d5e85)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:55.833342 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61667921-156b-4ca0-b64c-73f5e5eddc14 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:55.896341 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-61667921-156b-4ca0-b64c-73f5e5eddc14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d44dd7707a45399e078892328d5e85', 'request_id': 'req-61667921-156b-4ca0-b64c-73f5e5eddc14'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:55.905112 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-61667921-156b-4ca0-b64c-73f5e5eddc14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d44dd7707a45399e078892328d5e85', 'request_id': 'req-61667921-156b-4ca0-b64c-73f5e5eddc14'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:55.914873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-61667921-156b-4ca0-b64c-73f5e5eddc14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d44dd7707a45399e078892328d5e85', 'request_id': 'req-61667921-156b-4ca0-b64c-73f5e5eddc14'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:55.930100 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-61667921-156b-4ca0-b64c-73f5e5eddc14 admin admin] The token cache is being invalidate because project 45d44dd7707a45399e078892328d5e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:55.930512 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-61667921-156b-4ca0-b64c-73f5e5eddc14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:55.931950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-61667921-156b-4ca0-b64c-73f5e5eddc14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:55.936084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 964/1936] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:55 2022] DELETE /identity/v3/projects/45d44dd7707a45399e078892328d5e85 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:55.941118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0ddff-ceae-4e76-b530-d70bad0302d0 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:55.953164 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0ddff-ceae-4e76-b530-d70bad0302d0 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:56.030975 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0cafd5-ed52-48ee-8a68-2561e3b3897d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:56.031932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cafd5-ed52-48ee-8a68-2561e3b3897d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:56.032142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cafd5-ed52-48ee-8a68-2561e3b3897d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:56.032403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cafd5-ed52-48ee-8a68-2561e3b3897d admin admin] PATH_INFO: `/v3/users/da061bce65274d7fbbbe38c3b8ccde24` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:56.033982 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0cafd5-ed52-48ee-8a68-2561e3b3897d admin admin] RBAC: Authorizing `identity:delete_user(user_id=da061bce65274d7fbbbe38c3b8ccde24)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:56.036477 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0cafd5-ed52-48ee-8a68-2561e3b3897d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:56.121205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1b0cafd5-ed52-48ee-8a68-2561e3b3897d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da061bce65274d7fbbbe38c3b8ccde24'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.130026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1b0cafd5-ed52-48ee-8a68-2561e3b3897d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da061bce65274d7fbbbe38c3b8ccde24'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.132014 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1b0cafd5-ed52-48ee-8a68-2561e3b3897d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da061bce65274d7fbbbe38c3b8ccde24'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.132738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1b0cafd5-ed52-48ee-8a68-2561e3b3897d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da061bce65274d7fbbbe38c3b8ccde24'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.142899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1b0cafd5-ed52-48ee-8a68-2561e3b3897d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da061bce65274d7fbbbe38c3b8ccde24'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.167553 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 973/1937] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:55 2022] DELETE /identity/v3/users/da061bce65274d7fbbbe38c3b8ccde24 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:56.273146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61667921-156b-4ca0-b64c-73f5e5eddc14 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:56.275628 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-61667921-156b-4ca0-b64c-73f5e5eddc14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:56.294371 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0cafd5-ed52-48ee-8a68-2561e3b3897d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:56.296659 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1b0cafd5-ed52-48ee-8a68-2561e3b3897d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:56.320872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61667921-156b-4ca0-b64c-73f5e5eddc14 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:56.338519 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:56.422304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0cafd5-ed52-48ee-8a68-2561e3b3897d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:56.429076 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784db67-5566-4bd9-9bbe-7715a41039bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:56.429919 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784db67-5566-4bd9-9bbe-7715a41039bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:56.430297 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784db67-5566-4bd9-9bbe-7715a41039bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:56.430672 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784db67-5566-4bd9-9bbe-7715a41039bf admin admin] PATH_INFO: `/v3/projects/b0278d42d0c04365998dbd2d14f13ee8` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:56.431965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784db67-5566-4bd9-9bbe-7715a41039bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0278d42d0c04365998dbd2d14f13ee8)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:56.434255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784db67-5566-4bd9-9bbe-7715a41039bf admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:56.469785 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396d121-921a-492d-88d3-f38d3ffeacbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:56.471038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396d121-921a-492d-88d3-f38d3ffeacbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:56.471771 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396d121-921a-492d-88d3-f38d3ffeacbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:56.472262 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396d121-921a-492d-88d3-f38d3ffeacbf admin admin] PATH_INFO: `/v3/projects/dd1c2d95526e4e8aba8fb7fd44eed029` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:56.473779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396d121-921a-492d-88d3-f38d3ffeacbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd1c2d95526e4e8aba8fb7fd44eed029)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:56.476002 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396d121-921a-492d-88d3-f38d3ffeacbf admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:56.544952 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b784db67-5566-4bd9-9bbe-7715a41039bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0278d42d0c04365998dbd2d14f13ee8', 'request_id': 'req-b784db67-5566-4bd9-9bbe-7715a41039bf'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.563369 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b784db67-5566-4bd9-9bbe-7715a41039bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0278d42d0c04365998dbd2d14f13ee8', 'request_id': 'req-b784db67-5566-4bd9-9bbe-7715a41039bf'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.577028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7396d121-921a-492d-88d3-f38d3ffeacbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd1c2d95526e4e8aba8fb7fd44eed029', 'request_id': 'req-7396d121-921a-492d-88d3-f38d3ffeacbf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.577411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b784db67-5566-4bd9-9bbe-7715a41039bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0278d42d0c04365998dbd2d14f13ee8', 'request_id': 'req-b784db67-5566-4bd9-9bbe-7715a41039bf'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.591540 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7396d121-921a-492d-88d3-f38d3ffeacbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd1c2d95526e4e8aba8fb7fd44eed029', 'request_id': 'req-7396d121-921a-492d-88d3-f38d3ffeacbf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.600336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b784db67-5566-4bd9-9bbe-7715a41039bf admin admin] The token cache is being invalidate because project b0278d42d0c04365998dbd2d14f13ee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:56.600939 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b784db67-5566-4bd9-9bbe-7715a41039bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.602479 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b784db67-5566-4bd9-9bbe-7715a41039bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.603749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7396d121-921a-492d-88d3-f38d3ffeacbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd1c2d95526e4e8aba8fb7fd44eed029', 'request_id': 'req-7396d121-921a-492d-88d3-f38d3ffeacbf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.610137 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 974/1938] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:56 2022] DELETE /identity/v3/projects/b0278d42d0c04365998dbd2d14f13ee8 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:56.610282 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:56.611546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784db67-5566-4bd9-9bbe-7715a41039bf admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:56.614812 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b784db67-5566-4bd9-9bbe-7715a41039bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:56.629996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7396d121-921a-492d-88d3-f38d3ffeacbf admin admin] The token cache is being invalidate because project dd1c2d95526e4e8aba8fb7fd44eed029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:56.630421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7396d121-921a-492d-88d3-f38d3ffeacbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.634174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-7396d121-921a-492d-88d3-f38d3ffeacbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.634174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 965/1939] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:56 2022] DELETE /identity/v3/projects/dd1c2d95526e4e8aba8fb7fd44eed029 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:56.638368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396d121-921a-492d-88d3-f38d3ffeacbf admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:56.644123 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7396d121-921a-492d-88d3-f38d3ffeacbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:56.722828 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784db67-5566-4bd9-9bbe-7715a41039bf admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:56.763062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396d121-921a-492d-88d3-f38d3ffeacbf admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:56.776265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a528c-24e2-480d-b470-721cba383c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:56.777511 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a528c-24e2-480d-b470-721cba383c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:56.777996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a528c-24e2-480d-b470-721cba383c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:56.778282 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a528c-24e2-480d-b470-721cba383c2e admin admin] PATH_INFO: `/v3/users/51a749bdaa444ddcadd9d265856ccd4f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:56.779680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a528c-24e2-480d-b470-721cba383c2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=51a749bdaa444ddcadd9d265856ccd4f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:56.781828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a528c-24e2-480d-b470-721cba383c2e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:56.814238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:56.830001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6e86c-5575-4273-9b35-457e9c70f865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:56.832362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6e86c-5575-4273-9b35-457e9c70f865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:56.833899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6e86c-5575-4273-9b35-457e9c70f865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:56.834193 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6e86c-5575-4273-9b35-457e9c70f865 admin admin] PATH_INFO: `/v3/users/7202bddd8c2647a59ec3d86d08f40069` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:56.835402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6e86c-5575-4273-9b35-457e9c70f865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7202bddd8c2647a59ec3d86d08f40069)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:56.837499 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6e86c-5575-4273-9b35-457e9c70f865 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:56.842660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-351a528c-24e2-480d-b470-721cba383c2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51a749bdaa444ddcadd9d265856ccd4f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.852319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-351a528c-24e2-480d-b470-721cba383c2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51a749bdaa444ddcadd9d265856ccd4f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.853611 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-351a528c-24e2-480d-b470-721cba383c2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51a749bdaa444ddcadd9d265856ccd4f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.854408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-351a528c-24e2-480d-b470-721cba383c2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51a749bdaa444ddcadd9d265856ccd4f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.865755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-351a528c-24e2-480d-b470-721cba383c2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51a749bdaa444ddcadd9d265856ccd4f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.890453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 966/1940] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:56 2022] DELETE /identity/v3/users/51a749bdaa444ddcadd9d265856ccd4f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:56.894343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a528c-24e2-480d-b470-721cba383c2e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:56.895856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-351a528c-24e2-480d-b470-721cba383c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:56.897413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d2c6e86c-5575-4273-9b35-457e9c70f865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7202bddd8c2647a59ec3d86d08f40069'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.910549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d2c6e86c-5575-4273-9b35-457e9c70f865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7202bddd8c2647a59ec3d86d08f40069'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.911314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d2c6e86c-5575-4273-9b35-457e9c70f865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7202bddd8c2647a59ec3d86d08f40069'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.912024 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d2c6e86c-5575-4273-9b35-457e9c70f865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7202bddd8c2647a59ec3d86d08f40069'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.936905 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d2c6e86c-5575-4273-9b35-457e9c70f865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7202bddd8c2647a59ec3d86d08f40069'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:56.960389 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 975/1941] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:56 2022] DELETE /identity/v3/users/7202bddd8c2647a59ec3d86d08f40069 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:56.982052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a528c-24e2-480d-b470-721cba383c2e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:57.007683 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:57.058495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422cd771-5ab8-46c3-89bf-730871c1e63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:57.059469 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cd771-5ab8-46c3-89bf-730871c1e63f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.059949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cd771-5ab8-46c3-89bf-730871c1e63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.060228 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cd771-5ab8-46c3-89bf-730871c1e63f admin admin] PATH_INFO: `/v3/projects/0b0b610ee7404339b4c1c5d8bf60d456` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.061416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422cd771-5ab8-46c3-89bf-730871c1e63f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b0b610ee7404339b4c1c5d8bf60d456)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:57.063560 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422cd771-5ab8-46c3-89bf-730871c1e63f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:57.120156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-422cd771-5ab8-46c3-89bf-730871c1e63f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b0b610ee7404339b4c1c5d8bf60d456', 'request_id': 'req-422cd771-5ab8-46c3-89bf-730871c1e63f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:57.129764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-422cd771-5ab8-46c3-89bf-730871c1e63f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b0b610ee7404339b4c1c5d8bf60d456', 'request_id': 'req-422cd771-5ab8-46c3-89bf-730871c1e63f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:57.137173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-422cd771-5ab8-46c3-89bf-730871c1e63f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b0b610ee7404339b4c1c5d8bf60d456', 'request_id': 'req-422cd771-5ab8-46c3-89bf-730871c1e63f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:57.148266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-422cd771-5ab8-46c3-89bf-730871c1e63f admin admin] The token cache is being invalidate because project 0b0b610ee7404339b4c1c5d8bf60d456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:57.148422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-422cd771-5ab8-46c3-89bf-730871c1e63f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:57.149494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-422cd771-5ab8-46c3-89bf-730871c1e63f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:57.152977 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 967/1942] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:56 2022] DELETE /identity/v3/projects/0b0b610ee7404339b4c1c5d8bf60d456 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:57.162429 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f4a07-0614-4409-ba24-4df6ebd39be9 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.162722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f4a07-0614-4409-ba24-4df6ebd39be9 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.163069 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f4a07-0614-4409-ba24-4df6ebd39be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.184010 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3f8f4a07-0614-4409-ba24-4df6ebd39be9 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:57.185240 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3f8f4a07-0614-4409-ba24-4df6ebd39be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:57.253186 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 976/1943] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:57.261766 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5a9af-af84-4b59-80d8-1c07c99b319c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.262045 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5a9af-af84-4b59-80d8-1c07c99b319c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.262313 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5a9af-af84-4b59-80d8-1c07c99b319c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.283699 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-08c5a9af-af84-4b59-80d8-1c07c99b319c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:57.285300 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-08c5a9af-af84-4b59-80d8-1c07c99b319c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:57.296253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 968/1944] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:57.300557 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f4a07-0614-4409-ba24-4df6ebd39be9 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:57.311167 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f4a07-0614-4409-ba24-4df6ebd39be9 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:57.317303 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa703544-fc79-4e33-90ba-006801db4665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:57.317935 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa703544-fc79-4e33-90ba-006801db4665 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.318219 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa703544-fc79-4e33-90ba-006801db4665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.318496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa703544-fc79-4e33-90ba-006801db4665 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.319158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa703544-fc79-4e33-90ba-006801db4665 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:57.319460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa703544-fc79-4e33-90ba-006801db4665 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:57.321100 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa703544-fc79-4e33-90ba-006801db4665 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:57.334494 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 977/1945] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:57.339216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5a9af-af84-4b59-80d8-1c07c99b319c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:57.353744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5a9af-af84-4b59-80d8-1c07c99b319c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:57.358523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c598d4-31e5-4e27-9ef5-f9e46b51e431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:57.360950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c598d4-31e5-4e27-9ef5-f9e46b51e431 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.360950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c598d4-31e5-4e27-9ef5-f9e46b51e431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.360950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c598d4-31e5-4e27-9ef5-f9e46b51e431 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.360950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c598d4-31e5-4e27-9ef5-f9e46b51e431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:57.362985 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c598d4-31e5-4e27-9ef5-f9e46b51e431 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:57.395993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 969/1946] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:57 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:22:57.400264 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa703544-fc79-4e33-90ba-006801db4665 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:57.412515 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa703544-fc79-4e33-90ba-006801db4665 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:57.412997 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c598d4-31e5-4e27-9ef5-f9e46b51e431 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:57.415212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-20c598d4-31e5-4e27-9ef5-f9e46b51e431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:57.490170 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a626c0-0de5-44e7-bf90-3fef88ccd2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:57.490170 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a626c0-0de5-44e7-bf90-3fef88ccd2a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.490170 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a626c0-0de5-44e7-bf90-3fef88ccd2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.490929 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a626c0-0de5-44e7-bf90-3fef88ccd2a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.494273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a626c0-0de5-44e7-bf90-3fef88ccd2a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:57.496341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a626c0-0de5-44e7-bf90-3fef88ccd2a4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:57.504876 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:57.521052 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 978/1947] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:57 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:57.521972 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a626c0-0de5-44e7-bf90-3fef88ccd2a4 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:57.526093 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-48a626c0-0de5-44e7-bf90-3fef88ccd2a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:57.530828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c598d4-31e5-4e27-9ef5-f9e46b51e431 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:57.537114 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe152df-597c-4076-b818-7bf1f286a592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:57.537921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe152df-597c-4076-b818-7bf1f286a592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.538501 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe152df-597c-4076-b818-7bf1f286a592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.538829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe152df-597c-4076-b818-7bf1f286a592 admin admin] PATH_INFO: `/v3/projects/f2301ca0dee846ca818a6a2ab98764c0` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.539886 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a626c0-0de5-44e7-bf90-3fef88ccd2a4 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:57.540025 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe152df-597c-4076-b818-7bf1f286a592 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2301ca0dee846ca818a6a2ab98764c0)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:57.541905 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe152df-597c-4076-b818-7bf1f286a592 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:57.545193 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588aaaa1-cfb3-4af1-a487-527f431b92fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:57.545831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aaaa1-cfb3-4af1-a487-527f431b92fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.546283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aaaa1-cfb3-4af1-a487-527f431b92fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.546563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aaaa1-cfb3-4af1-a487-527f431b92fd admin admin] PATH_INFO: `/v3/projects/d74251e7a24545c3bc82fe30518ffe57` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.547733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588aaaa1-cfb3-4af1-a487-527f431b92fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d74251e7a24545c3bc82fe30518ffe57)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:57.551303 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588aaaa1-cfb3-4af1-a487-527f431b92fd admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:57.575702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:57.608236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-588aaaa1-cfb3-4af1-a487-527f431b92fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd74251e7a24545c3bc82fe30518ffe57', 'request_id': 'req-588aaaa1-cfb3-4af1-a487-527f431b92fd'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:57.609617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-efe152df-597c-4076-b818-7bf1f286a592 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2301ca0dee846ca818a6a2ab98764c0', 'request_id': 'req-efe152df-597c-4076-b818-7bf1f286a592'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:57.620001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-588aaaa1-cfb3-4af1-a487-527f431b92fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd74251e7a24545c3bc82fe30518ffe57', 'request_id': 'req-588aaaa1-cfb3-4af1-a487-527f431b92fd'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:57.622328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-efe152df-597c-4076-b818-7bf1f286a592 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2301ca0dee846ca818a6a2ab98764c0', 'request_id': 'req-efe152df-597c-4076-b818-7bf1f286a592'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:57.627065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-588aaaa1-cfb3-4af1-a487-527f431b92fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd74251e7a24545c3bc82fe30518ffe57', 'request_id': 'req-588aaaa1-cfb3-4af1-a487-527f431b92fd'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:57.628899 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-efe152df-597c-4076-b818-7bf1f286a592 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2301ca0dee846ca818a6a2ab98764c0', 'request_id': 'req-efe152df-597c-4076-b818-7bf1f286a592'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:57.640571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-efe152df-597c-4076-b818-7bf1f286a592 admin admin] The token cache is being invalidate because project f2301ca0dee846ca818a6a2ab98764c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:57.640861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-efe152df-597c-4076-b818-7bf1f286a592 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:57.641542 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-efe152df-597c-4076-b818-7bf1f286a592 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:57.641760 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-588aaaa1-cfb3-4af1-a487-527f431b92fd admin admin] The token cache is being invalidate because project d74251e7a24545c3bc82fe30518ffe57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:57.642055 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-588aaaa1-cfb3-4af1-a487-527f431b92fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:57.642721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-588aaaa1-cfb3-4af1-a487-527f431b92fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:57.643062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 970/1948] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:57 2022] DELETE /identity/v3/projects/f2301ca0dee846ca818a6a2ab98764c0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:57.644174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe152df-597c-4076-b818-7bf1f286a592 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:57.644926 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 979/1949] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:57 2022] DELETE /identity/v3/projects/d74251e7a24545c3bc82fe30518ffe57 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:57.645683 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-efe152df-597c-4076-b818-7bf1f286a592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:57.649336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c77c6-27d8-46d2-a0c3-8985ac8d6c05 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.649601 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c77c6-27d8-46d2-a0c3-8985ac8d6c05 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.649925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c77c6-27d8-46d2-a0c3-8985ac8d6c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.672859 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fe7c77c6-27d8-46d2-a0c3-8985ac8d6c05 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:57.674100 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fe7c77c6-27d8-46d2-a0c3-8985ac8d6c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:57.716165 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe152df-597c-4076-b818-7bf1f286a592 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:57.749094 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 980/1950] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:57.753505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9a575-3a4c-4c35-88d8-6f51f7e135b8 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.753505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9a575-3a4c-4c35-88d8-6f51f7e135b8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.753505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9a575-3a4c-4c35-88d8-6f51f7e135b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.778110 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-cdc9a575-3a4c-4c35-88d8-6f51f7e135b8 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:57.779343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cdc9a575-3a4c-4c35-88d8-6f51f7e135b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:57.793912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 981/1951] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:57.795185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc228dab-547a-4ba1-b2b0-4a002658fe99 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.795473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc228dab-547a-4ba1-b2b0-4a002658fe99 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.795777 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc228dab-547a-4ba1-b2b0-4a002658fe99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.825187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8747b7-7d91-4174-9be5-7f4288a43c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:57.825919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8747b7-7d91-4174-9be5-7f4288a43c28 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.826278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8747b7-7d91-4174-9be5-7f4288a43c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.826606 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8747b7-7d91-4174-9be5-7f4288a43c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.827279 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8747b7-7d91-4174-9be5-7f4288a43c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:57.828330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-bc228dab-547a-4ba1-b2b0-4a002658fe99 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:57.829181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8747b7-7d91-4174-9be5-7f4288a43c28 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:57.829534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bc228dab-547a-4ba1-b2b0-4a002658fe99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:57.838445 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 982/1952] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:57.839774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3626c26-3952-4812-9a57-e9315690b8c3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.840057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3626c26-3952-4812-9a57-e9315690b8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.840328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3626c26-3952-4812-9a57-e9315690b8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.840449 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 971/1953] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:57.844712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8747b7-7d91-4174-9be5-7f4288a43c28 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:57.856544 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8747b7-7d91-4174-9be5-7f4288a43c28 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:57.861421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c0ed3-6fde-4c20-9f5a-71da8e7ea948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:57.862031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c0ed3-6fde-4c20-9f5a-71da8e7ea948 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.863052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c0ed3-6fde-4c20-9f5a-71da8e7ea948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.863052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c0ed3-6fde-4c20-9f5a-71da8e7ea948 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.863268 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c0ed3-6fde-4c20-9f5a-71da8e7ea948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:57.863495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c0ed3-6fde-4c20-9f5a-71da8e7ea948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:57.865106 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c0ed3-6fde-4c20-9f5a-71da8e7ea948 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:57.869128 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d3626c26-3952-4812-9a57-e9315690b8c3 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:57.870426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d3626c26-3952-4812-9a57-e9315690b8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:57.881373 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 972/1954] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:57.881373 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c0ed3-6fde-4c20-9f5a-71da8e7ea948 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:57.881621 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 983/1955] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:57.881698 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f08c0ed3-6fde-4c20-9f5a-71da8e7ea948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:57.882466 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3626c26-3952-4812-9a57-e9315690b8c3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:57.894846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3626c26-3952-4812-9a57-e9315690b8c3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:57.897738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c0ed3-6fde-4c20-9f5a-71da8e7ea948 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:57.899542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e53d6e-2890-44fe-81d2-a51f06670108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:57.900153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53d6e-2890-44fe-81d2-a51f06670108 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.900438 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53d6e-2890-44fe-81d2-a51f06670108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.900712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53d6e-2890-44fe-81d2-a51f06670108 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.901401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e53d6e-2890-44fe-81d2-a51f06670108 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:57.902994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1268dc-778a-44b4-9ac0-f0d063f3b6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:57.903633 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1268dc-778a-44b4-9ac0-f0d063f3b6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.903865 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e53d6e-2890-44fe-81d2-a51f06670108 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:57.903985 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1268dc-778a-44b4-9ac0-f0d063f3b6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.904206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1268dc-778a-44b4-9ac0-f0d063f3b6d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.904889 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1268dc-778a-44b4-9ac0-f0d063f3b6d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:57.906467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1268dc-778a-44b4-9ac0-f0d063f3b6d6 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:57.917326 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 973/1956] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:57.917975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1268dc-778a-44b4-9ac0-f0d063f3b6d6 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:57.929267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1268dc-778a-44b4-9ac0-f0d063f3b6d6 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:57.936321 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f55d2-560f-48f1-a90e-b74ab1fbf4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:57.936947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f55d2-560f-48f1-a90e-b74ab1fbf4cc admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:57.941277 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 984/1957] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:57 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:57.941969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f55d2-560f-48f1-a90e-b74ab1fbf4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:57.941969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f55d2-560f-48f1-a90e-b74ab1fbf4cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:57.942131 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e53d6e-2890-44fe-81d2-a51f06670108 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:57.942204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f55d2-560f-48f1-a90e-b74ab1fbf4cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:22:57.942204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f55d2-560f-48f1-a90e-b74ab1fbf4cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:57.942204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f55d2-560f-48f1-a90e-b74ab1fbf4cc admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:57.956169 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 974/1958] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:22:57.956444 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f55d2-560f-48f1-a90e-b74ab1fbf4cc admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:57.969701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f55d2-560f-48f1-a90e-b74ab1fbf4cc admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:57.983204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:57.985122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:58.018953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e53d6e-2890-44fe-81d2-a51f06670108 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:58.023540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a94f6-4a88-4e34-bdf3-7e86630cef69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:58.023913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a94f6-4a88-4e34-bdf3-7e86630cef69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.024198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a94f6-4a88-4e34-bdf3-7e86630cef69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.024588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a94f6-4a88-4e34-bdf3-7e86630cef69 admin admin] PATH_INFO: `/v3/projects/0610c3c8314f4212b848fd79761cf145/users/5881c80a14fa4c19aa06ee172f179ea4/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.046509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a94f6-4a88-4e34-bdf3-7e86630cef69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0610c3c8314f4212b848fd79761cf145, user_id=5881c80a14fa4c19aa06ee172f179ea4, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:58.049265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a94f6-4a88-4e34-bdf3-7e86630cef69 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:58.064565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 985/1959] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:57 2022] PUT /identity/v3/projects/0610c3c8314f4212b848fd79761cf145/users/5881c80a14fa4c19aa06ee172f179ea4/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:58.065356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a94f6-4a88-4e34-bdf3-7e86630cef69 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:58.066514 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e7610-79a2-4eea-a7dc-23cdc911eb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:58.067170 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e7610-79a2-4eea-a7dc-23cdc911eb15 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.067419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e7610-79a2-4eea-a7dc-23cdc911eb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.067729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e7610-79a2-4eea-a7dc-23cdc911eb15 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.068374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e7610-79a2-4eea-a7dc-23cdc911eb15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:58.070309 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e7610-79a2-4eea-a7dc-23cdc911eb15 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:58.092919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 975/1960] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:57 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:58.095056 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345efb3a-7000-4631-9c49-32067e7ebcb8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.095215 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345efb3a-7000-4631-9c49-32067e7ebcb8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.095498 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345efb3a-7000-4631-9c49-32067e7ebcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.124416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-345efb3a-7000-4631-9c49-32067e7ebcb8 None None] MFA Rules not processed for user `5881c80a14fa4c19aa06ee172f179ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:58.125638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-345efb3a-7000-4631-9c49-32067e7ebcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:58.135208 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a94f6-4a88-4e34-bdf3-7e86630cef69 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:58.140311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bdd33-e2b0-4382-8a87-708751409f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:58.141054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bdd33-e2b0-4382-8a87-708751409f92 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.141431 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bdd33-e2b0-4382-8a87-708751409f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.141777 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bdd33-e2b0-4382-8a87-708751409f92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.142480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bdd33-e2b0-4382-8a87-708751409f92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:58.144633 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bdd33-e2b0-4382-8a87-708751409f92 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:58.173237 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 986/1961] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:58 2022] POST /identity/v3/projects => generated 361 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:58.174068 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bdd33-e2b0-4382-8a87-708751409f92 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:58.190904 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 976/1962] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:58.191750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345efb3a-7000-4631-9c49-32067e7ebcb8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:58.202060 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345efb3a-7000-4631-9c49-32067e7ebcb8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:58.240359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bdd33-e2b0-4382-8a87-708751409f92 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:58.245183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8e263-2e8e-42d5-998a-be5d6328c258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:58.246330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8e263-2e8e-42d5-998a-be5d6328c258 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.246330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8e263-2e8e-42d5-998a-be5d6328c258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.246496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8e263-2e8e-42d5-998a-be5d6328c258 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.247020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8e263-2e8e-42d5-998a-be5d6328c258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:58.248658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8e263-2e8e-42d5-998a-be5d6328c258 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:58.259018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 987/1963] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:58.259796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8e263-2e8e-42d5-998a-be5d6328c258 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:58.261791 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-49d8e263-2e8e-42d5-998a-be5d6328c258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:58.266460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22670591-9db3-4461-a598-875957c9eef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:58.267084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22670591-9db3-4461-a598-875957c9eef4 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.267370 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22670591-9db3-4461-a598-875957c9eef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.267679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22670591-9db3-4461-a598-875957c9eef4 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.268780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22670591-9db3-4461-a598-875957c9eef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:58.270259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22670591-9db3-4461-a598-875957c9eef4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:58.292565 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 977/1964] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:58 2022] POST /identity/v3/users => generated 281 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:58.293475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22670591-9db3-4461-a598-875957c9eef4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:58.303149 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22670591-9db3-4461-a598-875957c9eef4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:58.303705 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:58.308918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2840a-b9d7-4643-b6cb-00bbad953065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:58.309577 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2840a-b9d7-4643-b6cb-00bbad953065 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.309916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2840a-b9d7-4643-b6cb-00bbad953065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.310308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2840a-b9d7-4643-b6cb-00bbad953065 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.310953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf2840a-b9d7-4643-b6cb-00bbad953065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:58.312609 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf2840a-b9d7-4643-b6cb-00bbad953065 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:58.322580 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 978/1965] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:58.323035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8e263-2e8e-42d5-998a-be5d6328c258 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:58.323492 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2840a-b9d7-4643-b6cb-00bbad953065 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:58.327114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e71958-a934-4575-9403-3399ff6d9d20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:58.327762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e71958-a934-4575-9403-3399ff6d9d20 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.328048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e71958-a934-4575-9403-3399ff6d9d20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.328325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e71958-a934-4575-9403-3399ff6d9d20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.335052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2840a-b9d7-4643-b6cb-00bbad953065 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:58.340543 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c37cd3-de8d-4353-a6e6-341b295593f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:58.341154 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c37cd3-de8d-4353-a6e6-341b295593f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.341432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c37cd3-de8d-4353-a6e6-341b295593f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.341702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c37cd3-de8d-4353-a6e6-341b295593f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.342490 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c37cd3-de8d-4353-a6e6-341b295593f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:58.344411 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c37cd3-de8d-4353-a6e6-341b295593f8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:58.354786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 979/1966] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:58.355576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c37cd3-de8d-4353-a6e6-341b295593f8 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:58.361305 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:58.365400 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c37cd3-de8d-4353-a6e6-341b295593f8 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:58.372099 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5b511-9dda-4c0f-b95b-88838274e0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:58.372904 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5b511-9dda-4c0f-b95b-88838274e0ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.373551 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5b511-9dda-4c0f-b95b-88838274e0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.373821 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5b511-9dda-4c0f-b95b-88838274e0ce admin admin] PATH_INFO: `/v3/projects/2a97e649b405464a9fc7ec379401ca2c/users/8ef1871bd4154fe18b167d1b69f2decd/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.389456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5b511-9dda-4c0f-b95b-88838274e0ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a97e649b405464a9fc7ec379401ca2c, user_id=8ef1871bd4154fe18b167d1b69f2decd, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:58.389806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e71958-a934-4575-9403-3399ff6d9d20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:58.391266 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e71958-a934-4575-9403-3399ff6d9d20 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:58.392179 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5b511-9dda-4c0f-b95b-88838274e0ce admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:58.400397 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 988/1967] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:58.401817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e71958-a934-4575-9403-3399ff6d9d20 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:58.406534 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 980/1968] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:58 2022] PUT /identity/v3/projects/2a97e649b405464a9fc7ec379401ca2c/users/8ef1871bd4154fe18b167d1b69f2decd/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:58.412336 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88f4e6-fcdc-4b37-b7b9-7fc6990ecf51 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.413002 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88f4e6-fcdc-4b37-b7b9-7fc6990ecf51 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.413510 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88f4e6-fcdc-4b37-b7b9-7fc6990ecf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.413618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e71958-a934-4575-9403-3399ff6d9d20 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:58.438570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2e88f4e6-fcdc-4b37-b7b9-7fc6990ecf51 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:58.440057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2e88f4e6-fcdc-4b37-b7b9-7fc6990ecf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:58.476687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9edce-3572-435b-b551-c83090e7e523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:58.477182 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9edce-3572-435b-b551-c83090e7e523 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.477467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9edce-3572-435b-b551-c83090e7e523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.477740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9edce-3572-435b-b551-c83090e7e523 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.478383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9edce-3572-435b-b551-c83090e7e523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:58.480539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9edce-3572-435b-b551-c83090e7e523 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:58.491751 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 989/1969] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:22:58.495121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9edce-3572-435b-b551-c83090e7e523 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:58.506157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9edce-3572-435b-b551-c83090e7e523 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:58.509352 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 981/1970] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:58.514131 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e12dd-8810-40c4-99cc-1315537a22b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:58.514866 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e12dd-8810-40c4-99cc-1315537a22b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.515156 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e12dd-8810-40c4-99cc-1315537a22b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.515433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e12dd-8810-40c4-99cc-1315537a22b0 admin admin] PATH_INFO: `/v3/projects/f54196a225b4406fbbf263cbabdc1605/users/84331e88c09e4b37b993f8abb14be500/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.520980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88f4e6-fcdc-4b37-b7b9-7fc6990ecf51 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:58.522706 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2e88f4e6-fcdc-4b37-b7b9-7fc6990ecf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:58.535443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e12dd-8810-40c4-99cc-1315537a22b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f54196a225b4406fbbf263cbabdc1605, user_id=84331e88c09e4b37b993f8abb14be500, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:58.538348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e12dd-8810-40c4-99cc-1315537a22b0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:58.557323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 990/1971] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:22:58 2022] PUT /identity/v3/projects/f54196a225b4406fbbf263cbabdc1605/users/84331e88c09e4b37b993f8abb14be500/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:22:58.568701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbef166-01fb-444f-ac15-baa8e7332221 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.569140 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbef166-01fb-444f-ac15-baa8e7332221 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.569659 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbef166-01fb-444f-ac15-baa8e7332221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.606569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ffbef166-01fb-444f-ac15-baa8e7332221 None None] MFA Rules not processed for user `84331e88c09e4b37b993f8abb14be500`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:58.607888 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ffbef166-01fb-444f-ac15-baa8e7332221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:58.614001 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88f4e6-fcdc-4b37-b7b9-7fc6990ecf51 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:58.671528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:58.699202 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04152ee-fbe8-4e84-87e0-fb06a13e77ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:58.700330 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04152ee-fbe8-4e84-87e0-fb06a13e77ea service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.700620 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04152ee-fbe8-4e84-87e0-fb06a13e77ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.701045 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04152ee-fbe8-4e84-87e0-fb06a13e77ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.701684 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 991/1972] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:22:58.739320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbef166-01fb-444f-ac15-baa8e7332221 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:58.741187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ffbef166-01fb-444f-ac15-baa8e7332221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:58.785248 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:58.806621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04152ee-fbe8-4e84-87e0-fb06a13e77ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:58.808186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04152ee-fbe8-4e84-87e0-fb06a13e77ea service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:58.820956 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbef166-01fb-444f-ac15-baa8e7332221 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:58.825229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 982/1973] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:22:58.830244 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061f149-17ce-4891-ae83-c02b12725534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:58.832165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061f149-17ce-4891-ae83-c02b12725534 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.832313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061f149-17ce-4891-ae83-c02b12725534 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.832689 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061f149-17ce-4891-ae83-c02b12725534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.842849 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5061f149-17ce-4891-ae83-c02b12725534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:58.844457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5061f149-17ce-4891-ae83-c02b12725534 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:58.854717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 992/1974] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:22:58.871049 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04152ee-fbe8-4e84-87e0-fb06a13e77ea service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:58.881297 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04152ee-fbe8-4e84-87e0-fb06a13e77ea service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:58.887992 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f843e-f0c8-4d37-9dee-250f43224de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:58.888634 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f843e-f0c8-4d37-9dee-250f43224de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.888920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f843e-f0c8-4d37-9dee-250f43224de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.889193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f843e-f0c8-4d37-9dee-250f43224de1 admin admin] PATH_INFO: `/v3/users/84331e88c09e4b37b993f8abb14be500` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:58.890455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f843e-f0c8-4d37-9dee-250f43224de1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84331e88c09e4b37b993f8abb14be500)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:58.892495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f843e-f0c8-4d37-9dee-250f43224de1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:58.947238 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fc4f843e-f0c8-4d37-9dee-250f43224de1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84331e88c09e4b37b993f8abb14be500'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:58.957382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fc4f843e-f0c8-4d37-9dee-250f43224de1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84331e88c09e4b37b993f8abb14be500'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:58.958118 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fc4f843e-f0c8-4d37-9dee-250f43224de1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84331e88c09e4b37b993f8abb14be500'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:58.958756 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fc4f843e-f0c8-4d37-9dee-250f43224de1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84331e88c09e4b37b993f8abb14be500'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:58.966862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fc4f843e-f0c8-4d37-9dee-250f43224de1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84331e88c09e4b37b993f8abb14be500'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:58.985826 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 983/1975] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:22:58 2022] DELETE /identity/v3/users/84331e88c09e4b37b993f8abb14be500 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:58.990504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7e795-2116-46e6-8777-86464c76d4f5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:58.990885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7e795-2116-46e6-8777-86464c76d4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:58.991227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7e795-2116-46e6-8777-86464c76d4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:59.012537 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-88d7e795-2116-46e6-8777-86464c76d4f5 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:59.013750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-88d7e795-2116-46e6-8777-86464c76d4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:59.079227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 993/1976] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:59.090274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f843e-f0c8-4d37-9dee-250f43224de1 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:59.091727 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fc4f843e-f0c8-4d37-9dee-250f43224de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:59.161922 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f843e-f0c8-4d37-9dee-250f43224de1 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:59.166368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c83ca-c8a1-42d3-84a0-a23d01f56e20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:59.167136 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c83ca-c8a1-42d3-84a0-a23d01f56e20 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:59.167326 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c83ca-c8a1-42d3-84a0-a23d01f56e20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:59.168410 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c83ca-c8a1-42d3-84a0-a23d01f56e20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:59.178327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c83ca-c8a1-42d3-84a0-a23d01f56e20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:59.179818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c83ca-c8a1-42d3-84a0-a23d01f56e20 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:59.191878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 984/1977] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:22:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:22:59.575498 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7e795-2116-46e6-8777-86464c76d4f5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:59.577222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-88d7e795-2116-46e6-8777-86464c76d4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:59.590342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7e795-2116-46e6-8777-86464c76d4f5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:59.595759 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb721fa5-85ae-44d7-92c5-02eb7167338f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:59.596591 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb721fa5-85ae-44d7-92c5-02eb7167338f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:59.596926 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb721fa5-85ae-44d7-92c5-02eb7167338f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:59.597306 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb721fa5-85ae-44d7-92c5-02eb7167338f admin admin] PATH_INFO: `/v3/projects/f54196a225b4406fbbf263cbabdc1605` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:59.598534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb721fa5-85ae-44d7-92c5-02eb7167338f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f54196a225b4406fbbf263cbabdc1605)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:59.600742 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb721fa5-85ae-44d7-92c5-02eb7167338f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:59.666379 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cb721fa5-85ae-44d7-92c5-02eb7167338f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f54196a225b4406fbbf263cbabdc1605', 'request_id': 'req-cb721fa5-85ae-44d7-92c5-02eb7167338f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:59.681395 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cb721fa5-85ae-44d7-92c5-02eb7167338f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f54196a225b4406fbbf263cbabdc1605', 'request_id': 'req-cb721fa5-85ae-44d7-92c5-02eb7167338f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:59.698506 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6efffc-3087-43cd-8854-14c4454d3a64 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:59.698899 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6efffc-3087-43cd-8854-14c4454d3a64 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:59.699277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6efffc-3087-43cd-8854-14c4454d3a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:59.705922 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cb721fa5-85ae-44d7-92c5-02eb7167338f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f54196a225b4406fbbf263cbabdc1605', 'request_id': 'req-cb721fa5-85ae-44d7-92c5-02eb7167338f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:59.739784 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cb721fa5-85ae-44d7-92c5-02eb7167338f admin admin] The token cache is being invalidate because project f54196a225b4406fbbf263cbabdc1605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:22:59.740091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cb721fa5-85ae-44d7-92c5-02eb7167338f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:59.741422 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cb721fa5-85ae-44d7-92c5-02eb7167338f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:22:59.747650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 994/1978] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:22:59 2022] DELETE /identity/v3/projects/f54196a225b4406fbbf263cbabdc1605 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:22:59.759669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7c6efffc-3087-43cd-8854-14c4454d3a64 None None] MFA Rules not processed for user `8ef1871bd4154fe18b167d1b69f2decd`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:59.761685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7c6efffc-3087-43cd-8854-14c4454d3a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:59.761979 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a62da-8170-4797-86b9-808c4db4efd3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:59.765665 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a62da-8170-4797-86b9-808c4db4efd3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:59.766242 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a62da-8170-4797-86b9-808c4db4efd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:59.793044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-7c5a62da-8170-4797-86b9-808c4db4efd3 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:59.794888 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7c5a62da-8170-4797-86b9-808c4db4efd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:59.809585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:59.856339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:22:59.887140 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 985/1979] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:59.887736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6efffc-3087-43cd-8854-14c4454d3a64 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:59.889145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7c6efffc-3087-43cd-8854-14c4454d3a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:59.897853 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 995/1980] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:59.900224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d01db-61c2-44b1-a56f-3fef6c074d30 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:59.902153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d01db-61c2-44b1-a56f-3fef6c074d30 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:59.902634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d01db-61c2-44b1-a56f-3fef6c074d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:59.914054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6efffc-3087-43cd-8854-14c4454d3a64 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:59.921554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421880a6-2a84-4b65-b96e-e1df8f1cd443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:22:59.922192 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421880a6-2a84-4b65-b96e-e1df8f1cd443 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:22:59.922475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421880a6-2a84-4b65-b96e-e1df8f1cd443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:22:59.922749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421880a6-2a84-4b65-b96e-e1df8f1cd443 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:22:59.923395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421880a6-2a84-4b65-b96e-e1df8f1cd443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:22:59.925991 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421880a6-2a84-4b65-b96e-e1df8f1cd443 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:22:59.955153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-cf6d01db-61c2-44b1-a56f-3fef6c074d30 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:22:59.956421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cf6d01db-61c2-44b1-a56f-3fef6c074d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:22:59.976874 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 996/1981] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:22:59.976874 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d01db-61c2-44b1-a56f-3fef6c074d30 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:22:59.986508 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d01db-61c2-44b1-a56f-3fef6c074d30 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:22:59.987821 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 986/1982] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:22:59 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:22:59.990865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421880a6-2a84-4b65-b96e-e1df8f1cd443 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:00.057476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0bca3b-ae0a-43fb-97f3-f385d4d5081f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:00.058037 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bca3b-ae0a-43fb-97f3-f385d4d5081f admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:00.058317 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bca3b-ae0a-43fb-97f3-f385d4d5081f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:00.058591 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bca3b-ae0a-43fb-97f3-f385d4d5081f admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:00.059234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0bca3b-ae0a-43fb-97f3-f385d4d5081f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:23:00.059572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0bca3b-ae0a-43fb-97f3-f385d4d5081f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:00.061203 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0bca3b-ae0a-43fb-97f3-f385d4d5081f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:00.078704 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 997/1983] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:23:00.083536 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421880a6-2a84-4b65-b96e-e1df8f1cd443 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:00.084947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0bca3b-ae0a-43fb-97f3-f385d4d5081f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:00.088365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38b566-3b62-4115-83f8-f33b4c0e72de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:00.089412 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38b566-3b62-4115-83f8-f33b4c0e72de admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:00.089412 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38b566-3b62-4115-83f8-f33b4c0e72de admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:00.089787 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38b566-3b62-4115-83f8-f33b4c0e72de admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:00.090603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38b566-3b62-4115-83f8-f33b4c0e72de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:00.093520 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38b566-3b62-4115-83f8-f33b4c0e72de admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:00.095283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0bca3b-ae0a-43fb-97f3-f385d4d5081f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:00.100257 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3250c8-015c-4115-9fb6-580efad49ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:00.100864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3250c8-015c-4115-9fb6-580efad49ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:00.101147 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3250c8-015c-4115-9fb6-580efad49ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:00.101419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3250c8-015c-4115-9fb6-580efad49ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:00.102049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3250c8-015c-4115-9fb6-580efad49ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:00.104007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3250c8-015c-4115-9fb6-580efad49ec5 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:00.125220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 987/1984] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:22:59 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:00.126537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38b566-3b62-4115-83f8-f33b4c0e72de admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:00.136456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 998/1985] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:23:00 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:23:00.136660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38b566-3b62-4115-83f8-f33b4c0e72de admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:00.140171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3250c8-015c-4115-9fb6-580efad49ec5 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:00.213288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a6f24-bbc9-48de-8fe8-e4f31027ca57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:00.213998 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a6f24-bbc9-48de-8fe8-e4f31027ca57 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:00.214284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a6f24-bbc9-48de-8fe8-e4f31027ca57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:00.214560 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a6f24-bbc9-48de-8fe8-e4f31027ca57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:00.215361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a6f24-bbc9-48de-8fe8-e4f31027ca57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:00.218089 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a6f24-bbc9-48de-8fe8-e4f31027ca57 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:00.221450 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3250c8-015c-4115-9fb6-580efad49ec5 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:00.226929 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e7668-13ea-474d-986a-f932733ef0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:00.227838 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e7668-13ea-474d-986a-f932733ef0b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:00.228278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e7668-13ea-474d-986a-f932733ef0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:00.229145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e7668-13ea-474d-986a-f932733ef0b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:00.230428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e7668-13ea-474d-986a-f932733ef0b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:00.233040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e7668-13ea-474d-986a-f932733ef0b3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:00.237616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 988/1986] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:23:00.243644 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a6f24-bbc9-48de-8fe8-e4f31027ca57 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:00.267931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 999/1987] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:23:00 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:23:00.269512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e7668-13ea-474d-986a-f932733ef0b3 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:00.270813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a6f24-bbc9-48de-8fe8-e4f31027ca57 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:00.276213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67eb64f-ccdb-45dc-a408-358a9db015e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:00.277530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67eb64f-ccdb-45dc-a408-358a9db015e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:00.277714 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67eb64f-ccdb-45dc-a408-358a9db015e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:00.277986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67eb64f-ccdb-45dc-a408-358a9db015e2 admin admin] PATH_INFO: `/v3/projects/71b5f1c8eb874fc2924e02f0950d0fed/users/e39a28e74bfe43d1896e5d1f3af8d205/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:00.281650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e7668-13ea-474d-986a-f932733ef0b3 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:00.286750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241cf7a-06b3-4f38-84e4-d7a01052abef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:00.287471 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241cf7a-06b3-4f38-84e4-d7a01052abef admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:00.288021 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241cf7a-06b3-4f38-84e4-d7a01052abef admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:00.288401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241cf7a-06b3-4f38-84e4-d7a01052abef admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:00.289535 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241cf7a-06b3-4f38-84e4-d7a01052abef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:00.291785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241cf7a-06b3-4f38-84e4-d7a01052abef admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:00.296567 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67eb64f-ccdb-45dc-a408-358a9db015e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b5f1c8eb874fc2924e02f0950d0fed, user_id=e39a28e74bfe43d1896e5d1f3af8d205, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:00.299296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67eb64f-ccdb-45dc-a408-358a9db015e2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:00.305649 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1000/1988] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:00.307040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241cf7a-06b3-4f38-84e4-d7a01052abef admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:00.313305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:00.316957 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 989/1989] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:00 2022] PUT /identity/v3/projects/71b5f1c8eb874fc2924e02f0950d0fed/users/e39a28e74bfe43d1896e5d1f3af8d205/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:00.320097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241cf7a-06b3-4f38-84e4-d7a01052abef admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:00.320833 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67eb64f-ccdb-45dc-a408-358a9db015e2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:00.360567 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:00.390779 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89417d5-f762-401e-9da7-baabef58bab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:00.391552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89417d5-f762-401e-9da7-baabef58bab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:00.391850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89417d5-f762-401e-9da7-baabef58bab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:00.392129 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89417d5-f762-401e-9da7-baabef58bab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:00.392775 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89417d5-f762-401e-9da7-baabef58bab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:00.394363 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89417d5-f762-401e-9da7-baabef58bab2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:00.403879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67eb64f-ccdb-45dc-a408-358a9db015e2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:00.407614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1001/1990] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:23:00.409412 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20465902-5274-44c1-90ff-6ff42fc379b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:00.410161 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89417d5-f762-401e-9da7-baabef58bab2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:00.410408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20465902-5274-44c1-90ff-6ff42fc379b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:00.410690 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20465902-5274-44c1-90ff-6ff42fc379b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:00.410961 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20465902-5274-44c1-90ff-6ff42fc379b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:00.411667 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20465902-5274-44c1-90ff-6ff42fc379b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:00.413644 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20465902-5274-44c1-90ff-6ff42fc379b4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:00.422812 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89417d5-f762-401e-9da7-baabef58bab2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:00.425964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 990/1991] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:23:00.428758 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8c474-4330-4d5b-acf7-f08878b7bfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:00.429627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8c474-4330-4d5b-acf7-f08878b7bfcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:00.430324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8c474-4330-4d5b-acf7-f08878b7bfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:00.430462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20465902-5274-44c1-90ff-6ff42fc379b4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:00.431337 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8c474-4330-4d5b-acf7-f08878b7bfcc admin admin] PATH_INFO: `/v3/projects/3cac0cdaf48e4889919c48461db9a043/users/3d99b946d6d7488387fde76a3983a44f/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:00.448826 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20465902-5274-44c1-90ff-6ff42fc379b4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:00.451578 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8c474-4330-4d5b-acf7-f08878b7bfcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cac0cdaf48e4889919c48461db9a043, user_id=3d99b946d6d7488387fde76a3983a44f, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:00.452698 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8c474-4330-4d5b-acf7-f08878b7bfcc admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:00.454626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ded95-9f68-4f81-81bc-37e202b54f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:00.455301 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ded95-9f68-4f81-81bc-37e202b54f86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:00.455612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ded95-9f68-4f81-81bc-37e202b54f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:00.455885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ded95-9f68-4f81-81bc-37e202b54f86 admin admin] PATH_INFO: `/v3/projects/71b5f1c8eb874fc2924e02f0950d0fed/users/e39a28e74bfe43d1896e5d1f3af8d205/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:00.457964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ded95-9f68-4f81-81bc-37e202b54f86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b5f1c8eb874fc2924e02f0950d0fed, user_id=e39a28e74bfe43d1896e5d1f3af8d205, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:00.460937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ded95-9f68-4f81-81bc-37e202b54f86 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:00.467545 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1002/1992] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:00 2022] PUT /identity/v3/projects/3cac0cdaf48e4889919c48461db9a043/users/3d99b946d6d7488387fde76a3983a44f/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:00.475901 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 991/1993] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:00 2022] PUT /identity/v3/projects/71b5f1c8eb874fc2924e02f0950d0fed/users/e39a28e74bfe43d1896e5d1f3af8d205/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:00.478333 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f520f2b-3597-41d7-b89a-473d2112acce None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:00.478626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f520f2b-3597-41d7-b89a-473d2112acce None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:00.479107 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f520f2b-3597-41d7-b89a-473d2112acce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:00.511775 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3f520f2b-3597-41d7-b89a-473d2112acce None None] MFA Rules not processed for user `3d99b946d6d7488387fde76a3983a44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:00.513018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3f520f2b-3597-41d7-b89a-473d2112acce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:00.596629 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1003/1994] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:00.601290 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ded95-9f68-4f81-81bc-37e202b54f86 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:00.672352 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ded95-9f68-4f81-81bc-37e202b54f86 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:00.677517 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348f593-9c48-40c4-b035-988967b91205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:00.678204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348f593-9c48-40c4-b035-988967b91205 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:00.678558 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348f593-9c48-40c4-b035-988967b91205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:00.678898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348f593-9c48-40c4-b035-988967b91205 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:00.679642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348f593-9c48-40c4-b035-988967b91205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:00.681628 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348f593-9c48-40c4-b035-988967b91205 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:00.712231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 992/1995] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:23:00 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:00.716251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f520f2b-3597-41d7-b89a-473d2112acce None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:00.727488 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f520f2b-3597-41d7-b89a-473d2112acce None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:00.797392 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b6e10-0a55-4dea-934c-d672d5bd5560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:00.798118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b6e10-0a55-4dea-934c-d672d5bd5560 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:00.798295 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b6e10-0a55-4dea-934c-d672d5bd5560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:00.798719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b6e10-0a55-4dea-934c-d672d5bd5560 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:00.799224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b6e10-0a55-4dea-934c-d672d5bd5560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:00.801238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b6e10-0a55-4dea-934c-d672d5bd5560 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:00.813239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:00.830815 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1004/1996] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:23:00 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:00.835057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348f593-9c48-40c4-b035-988967b91205 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:00.845904 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348f593-9c48-40c4-b035-988967b91205 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:00.850712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b7af4-d7ed-460d-b497-484ca8744392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:00.851324 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b7af4-d7ed-460d-b497-484ca8744392 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:00.851641 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b7af4-d7ed-460d-b497-484ca8744392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:00.851930 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b7af4-d7ed-460d-b497-484ca8744392 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:00.852557 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b7af4-d7ed-460d-b497-484ca8744392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:00.854205 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b7af4-d7ed-460d-b497-484ca8744392 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:00.865972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 993/1997] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:00.870724 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b6e10-0a55-4dea-934c-d672d5bd5560 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:00.882832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b6e10-0a55-4dea-934c-d672d5bd5560 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:00.887792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22030b8-b253-49b4-bfa5-041cb653d0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:00.889449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22030b8-b253-49b4-bfa5-041cb653d0b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:00.889449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22030b8-b253-49b4-bfa5-041cb653d0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:00.889449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22030b8-b253-49b4-bfa5-041cb653d0b4 admin admin] PATH_INFO: `/v3/projects/225341fad2894e9e998ef6c07a8edc7e/users/4491fa00b1164692b5efe92cebee96a8/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:00.905924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22030b8-b253-49b4-bfa5-041cb653d0b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=225341fad2894e9e998ef6c07a8edc7e, user_id=4491fa00b1164692b5efe92cebee96a8, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:00.910590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22030b8-b253-49b4-bfa5-041cb653d0b4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:00.933187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1005/1998] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:00 2022] PUT /identity/v3/projects/225341fad2894e9e998ef6c07a8edc7e/users/4491fa00b1164692b5efe92cebee96a8/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:00.937300 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b7af4-d7ed-460d-b497-484ca8744392 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:00.948533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b7af4-d7ed-460d-b497-484ca8744392 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:01.029590 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619ade9-f7a9-46c8-9440-71be433a86ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:01.030221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ade9-f7a9-46c8-9440-71be433a86ce admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:01.030507 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ade9-f7a9-46c8-9440-71be433a86ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:01.031062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ade9-f7a9-46c8-9440-71be433a86ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:01.031496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619ade9-f7a9-46c8-9440-71be433a86ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:01.035180 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619ade9-f7a9-46c8-9440-71be433a86ce admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:01.047670 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 994/1999] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:01.052044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22030b8-b253-49b4-bfa5-041cb653d0b4 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:01.066916 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22030b8-b253-49b4-bfa5-041cb653d0b4 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:01.071818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2e2db-4166-46be-80f4-3d1e25789d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:01.072490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2e2db-4166-46be-80f4-3d1e25789d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:01.072779 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2e2db-4166-46be-80f4-3d1e25789d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:01.073076 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2e2db-4166-46be-80f4-3d1e25789d0d admin admin] PATH_INFO: `/v3/projects/225341fad2894e9e998ef6c07a8edc7e/users/4491fa00b1164692b5efe92cebee96a8/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:01.075036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf2e2db-4166-46be-80f4-3d1e25789d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=225341fad2894e9e998ef6c07a8edc7e, user_id=4491fa00b1164692b5efe92cebee96a8, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:01.077756 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf2e2db-4166-46be-80f4-3d1e25789d0d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:01.090105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1006/2000] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:01 2022] PUT /identity/v3/projects/225341fad2894e9e998ef6c07a8edc7e/users/4491fa00b1164692b5efe92cebee96a8/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:01.100791 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b7279-2db0-423a-9e31-7dee33a1c541 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:01.101117 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b7279-2db0-423a-9e31-7dee33a1c541 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:01.101395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b7279-2db0-423a-9e31-7dee33a1c541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:01.134423 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-864b7279-2db0-423a-9e31-7dee33a1c541 None None] MFA Rules not processed for user `4491fa00b1164692b5efe92cebee96a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:01.135761 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-864b7279-2db0-423a-9e31-7dee33a1c541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:01.148638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:01.213432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 995/2001] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:01.224968 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2e2db-4166-46be-80f4-3d1e25789d0d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:01.226687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bdf2e2db-4166-46be-80f4-3d1e25789d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:01.295023 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2e2db-4166-46be-80f4-3d1e25789d0d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:01.298699 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08b41c-0f41-42c3-9dfa-fc60164e336b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:01.299307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08b41c-0f41-42c3-9dfa-fc60164e336b service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:01.299612 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08b41c-0f41-42c3-9dfa-fc60164e336b service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:01.299883 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08b41c-0f41-42c3-9dfa-fc60164e336b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:01.363339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08b41c-0f41-42c3-9dfa-fc60164e336b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:01.365478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08b41c-0f41-42c3-9dfa-fc60164e336b service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:01.377317 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1007/2002] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:01.518480 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b25922-f6b8-4fea-a318-8f36c476834d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:01.518773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b25922-f6b8-4fea-a318-8f36c476834d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:01.519053 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b25922-f6b8-4fea-a318-8f36c476834d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:01.567118 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-71b25922-f6b8-4fea-a318-8f36c476834d None None] MFA Rules not processed for user `e39a28e74bfe43d1896e5d1f3af8d205`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:01.581904 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-71b25922-f6b8-4fea-a318-8f36c476834d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:01.668546 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 996/2003] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:01.682608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08b41c-0f41-42c3-9dfa-fc60164e336b service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:01.685383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9b08b41c-0f41-42c3-9dfa-fc60164e336b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:01.700267 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08b41c-0f41-42c3-9dfa-fc60164e336b service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:01.704615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac87c5-4004-44e1-8483-d963b5a2e211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:01.705294 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac87c5-4004-44e1-8483-d963b5a2e211 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:01.705583 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac87c5-4004-44e1-8483-d963b5a2e211 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:01.705864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac87c5-4004-44e1-8483-d963b5a2e211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:01.803008 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac87c5-4004-44e1-8483-d963b5a2e211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:01.804721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac87c5-4004-44e1-8483-d963b5a2e211 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:01.814773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1008/2004] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:04.820880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a84c-abfd-48e4-9098-bfc0596345a7 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:04.821448 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a84c-abfd-48e4-9098-bfc0596345a7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:04.821563 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a84c-abfd-48e4-9098-bfc0596345a7 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:04.823483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 997/2005] 10.222.0.33 () {54 vars in 921 bytes} [Thu Oct 6 12:23:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:23:04.832775 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd917373-9999-40e1-a39c-4c47eefb0c40 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:04.833247 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd917373-9999-40e1-a39c-4c47eefb0c40 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:04.833494 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd917373-9999-40e1-a39c-4c47eefb0c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:04.859875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-cd917373-9999-40e1-a39c-4c47eefb0c40 None None] MFA Rules not processed for user `5a46e63d34564d34b6b5752706375798`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:04.861257 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cd917373-9999-40e1-a39c-4c47eefb0c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:04.866026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:04.946953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1009/2006] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Oct 6 12:23:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:04.960616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a84c-abfd-48e4-9098-bfc0596345a7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:04.962473 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a115a84c-abfd-48e4-9098-bfc0596345a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:04.981508 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a84c-abfd-48e4-9098-bfc0596345a7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:04.987117 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f735a-cc1c-4fa1-b533-af8e177c1445 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:04.987721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f735a-cc1c-4fa1-b533-af8e177c1445 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:04.988007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f735a-cc1c-4fa1-b533-af8e177c1445 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:04.988280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f735a-cc1c-4fa1-b533-af8e177c1445 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:04.997486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f735a-cc1c-4fa1-b533-af8e177c1445 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:04.999154 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f735a-cc1c-4fa1-b533-af8e177c1445 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:05.017364 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 998/2007] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:05.310923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd917373-9999-40e1-a39c-4c47eefb0c40 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:05.320130 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd917373-9999-40e1-a39c-4c47eefb0c40 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:05.324305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead0665-c531-4386-a765-8c24e2cd96fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:05.324910 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead0665-c531-4386-a765-8c24e2cd96fa service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:05.325193 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead0665-c531-4386-a765-8c24e2cd96fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:05.325467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead0665-c531-4386-a765-8c24e2cd96fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:05.334679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead0665-c531-4386-a765-8c24e2cd96fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:05.336174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead0665-c531-4386-a765-8c24e2cd96fa service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:05.346699 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1010/2008] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:15.007955 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=63 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:15.342349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=33 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:16.901329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f735a-cc1c-4fa1-b533-af8e177c1445 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:16.981144 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f735a-cc1c-4fa1-b533-af8e177c1445 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:16.985838 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff5907-8747-4a0a-a940-e064a652e7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:16.986477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff5907-8747-4a0a-a940-e064a652e7b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:16.986760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff5907-8747-4a0a-a940-e064a652e7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:16.987035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff5907-8747-4a0a-a940-e064a652e7b0 admin admin] PATH_INFO: `/v3/users/3d99b946d6d7488387fde76a3983a44f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:16.988148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff5907-8747-4a0a-a940-e064a652e7b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d99b946d6d7488387fde76a3983a44f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:16.990434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff5907-8747-4a0a-a940-e064a652e7b0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:17.039077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-67ff5907-8747-4a0a-a940-e064a652e7b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d99b946d6d7488387fde76a3983a44f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:17.046970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-67ff5907-8747-4a0a-a940-e064a652e7b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d99b946d6d7488387fde76a3983a44f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:17.047703 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-67ff5907-8747-4a0a-a940-e064a652e7b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d99b946d6d7488387fde76a3983a44f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:17.048352 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-67ff5907-8747-4a0a-a940-e064a652e7b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d99b946d6d7488387fde76a3983a44f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:17.056360 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-67ff5907-8747-4a0a-a940-e064a652e7b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d99b946d6d7488387fde76a3983a44f'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:17.074226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 999/2009] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:23:16 2022] DELETE /identity/v3/users/3d99b946d6d7488387fde76a3983a44f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:23:17.078692 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c1422-ff76-4ce5-b4a8-96cdb43c4290 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:17.079166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c1422-ff76-4ce5-b4a8-96cdb43c4290 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:17.079282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c1422-ff76-4ce5-b4a8-96cdb43c4290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:17.099157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e12c1422-ff76-4ce5-b4a8-96cdb43c4290 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:17.100432 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e12c1422-ff76-4ce5-b4a8-96cdb43c4290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:17.173375 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1011/2010] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:23:17.184640 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff5907-8747-4a0a-a940-e064a652e7b0 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:17.185618 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-67ff5907-8747-4a0a-a940-e064a652e7b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:17.244466 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff5907-8747-4a0a-a940-e064a652e7b0 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:17.248425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d3d49-0304-4fb9-b9fc-f6ceec5f712e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:17.249045 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d3d49-0304-4fb9-b9fc-f6ceec5f712e service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:17.249330 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d3d49-0304-4fb9-b9fc-f6ceec5f712e service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:17.249606 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d3d49-0304-4fb9-b9fc-f6ceec5f712e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:17.260324 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d3d49-0304-4fb9-b9fc-f6ceec5f712e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:17.261777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d3d49-0304-4fb9-b9fc-f6ceec5f712e service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:17.271570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1000/2011] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:17.627522 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c1422-ff76-4ce5-b4a8-96cdb43c4290 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:17.628960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e12c1422-ff76-4ce5-b4a8-96cdb43c4290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:17.640893 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c1422-ff76-4ce5-b4a8-96cdb43c4290 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:17.646510 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce345436-1c10-4b4f-8650-7497a0dd95e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:17.647197 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce345436-1c10-4b4f-8650-7497a0dd95e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:17.647480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce345436-1c10-4b4f-8650-7497a0dd95e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:17.648049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce345436-1c10-4b4f-8650-7497a0dd95e8 admin admin] PATH_INFO: `/v3/projects/3cac0cdaf48e4889919c48461db9a043` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:17.649215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce345436-1c10-4b4f-8650-7497a0dd95e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cac0cdaf48e4889919c48461db9a043)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:17.651689 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce345436-1c10-4b4f-8650-7497a0dd95e8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:17.706763 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ce345436-1c10-4b4f-8650-7497a0dd95e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cac0cdaf48e4889919c48461db9a043', 'request_id': 'req-ce345436-1c10-4b4f-8650-7497a0dd95e8'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:17.717848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ce345436-1c10-4b4f-8650-7497a0dd95e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cac0cdaf48e4889919c48461db9a043', 'request_id': 'req-ce345436-1c10-4b4f-8650-7497a0dd95e8'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:17.725591 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ce345436-1c10-4b4f-8650-7497a0dd95e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cac0cdaf48e4889919c48461db9a043', 'request_id': 'req-ce345436-1c10-4b4f-8650-7497a0dd95e8'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:17.738260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ce345436-1c10-4b4f-8650-7497a0dd95e8 admin admin] The token cache is being invalidate because project 3cac0cdaf48e4889919c48461db9a043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:23:17.738551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ce345436-1c10-4b4f-8650-7497a0dd95e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:17.739202 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ce345436-1c10-4b4f-8650-7497a0dd95e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:17.745696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1012/2012] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:23:17 2022] DELETE /identity/v3/projects/3cac0cdaf48e4889919c48461db9a043 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:17.750105 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d3d49-0304-4fb9-b9fc-f6ceec5f712e service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:17.765820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d3d49-0304-4fb9-b9fc-f6ceec5f712e service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:17.809339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:17.890345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd38ec6-16cc-4403-9db5-14da1ec0d17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:17.891249 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd38ec6-16cc-4403-9db5-14da1ec0d17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:17.891599 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd38ec6-16cc-4403-9db5-14da1ec0d17c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:17.891725 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd38ec6-16cc-4403-9db5-14da1ec0d17c admin admin] PATH_INFO: `/v3/users/4491fa00b1164692b5efe92cebee96a8` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:17.892864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd38ec6-16cc-4403-9db5-14da1ec0d17c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4491fa00b1164692b5efe92cebee96a8)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:17.894851 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd38ec6-16cc-4403-9db5-14da1ec0d17c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:17.970187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cfd38ec6-16cc-4403-9db5-14da1ec0d17c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4491fa00b1164692b5efe92cebee96a8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:17.979003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cfd38ec6-16cc-4403-9db5-14da1ec0d17c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4491fa00b1164692b5efe92cebee96a8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:17.979954 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cfd38ec6-16cc-4403-9db5-14da1ec0d17c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4491fa00b1164692b5efe92cebee96a8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:17.980757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cfd38ec6-16cc-4403-9db5-14da1ec0d17c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4491fa00b1164692b5efe92cebee96a8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:17.990146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cfd38ec6-16cc-4403-9db5-14da1ec0d17c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4491fa00b1164692b5efe92cebee96a8'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:18.011583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1001/2013] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:23:17 2022] DELETE /identity/v3/users/4491fa00b1164692b5efe92cebee96a8 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:18.340731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce345436-1c10-4b4f-8650-7497a0dd95e8 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:18.342301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ce345436-1c10-4b4f-8650-7497a0dd95e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:18.356582 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce345436-1c10-4b4f-8650-7497a0dd95e8 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:18.422937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f69e6-f283-4278-aa40-d21912ea774f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:18.423499 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f69e6-f283-4278-aa40-d21912ea774f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:18.423810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f69e6-f283-4278-aa40-d21912ea774f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:18.424093 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f69e6-f283-4278-aa40-d21912ea774f admin admin] PATH_INFO: `/v3/projects/225341fad2894e9e998ef6c07a8edc7e` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:18.425215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f69e6-f283-4278-aa40-d21912ea774f admin admin] RBAC: Authorizing `identity:delete_project(project_id=225341fad2894e9e998ef6c07a8edc7e)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:18.427113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f69e6-f283-4278-aa40-d21912ea774f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:18.460999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:18.508195 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-955f69e6-f283-4278-aa40-d21912ea774f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '225341fad2894e9e998ef6c07a8edc7e', 'request_id': 'req-955f69e6-f283-4278-aa40-d21912ea774f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:18.518311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-955f69e6-f283-4278-aa40-d21912ea774f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '225341fad2894e9e998ef6c07a8edc7e', 'request_id': 'req-955f69e6-f283-4278-aa40-d21912ea774f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:18.528687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-955f69e6-f283-4278-aa40-d21912ea774f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '225341fad2894e9e998ef6c07a8edc7e', 'request_id': 'req-955f69e6-f283-4278-aa40-d21912ea774f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:18.554262 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-955f69e6-f283-4278-aa40-d21912ea774f admin admin] The token cache is being invalidate because project 225341fad2894e9e998ef6c07a8edc7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:23:18.554560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-955f69e6-f283-4278-aa40-d21912ea774f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:18.555238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-955f69e6-f283-4278-aa40-d21912ea774f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:18.560802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1013/2014] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:23:18 2022] DELETE /identity/v3/projects/225341fad2894e9e998ef6c07a8edc7e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:23:18.564783 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5cd77-7148-435b-a40f-a3deb84a6da4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:18.565168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5cd77-7148-435b-a40f-a3deb84a6da4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:18.565537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5cd77-7148-435b-a40f-a3deb84a6da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:18.588630 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-97d5cd77-7148-435b-a40f-a3deb84a6da4 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:18.589999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-97d5cd77-7148-435b-a40f-a3deb84a6da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:18.651787 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1002/2015] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:18.659829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71342575-3499-40d0-b328-2789f0c270d5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:18.660121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71342575-3499-40d0-b328-2789f0c270d5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:18.660426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71342575-3499-40d0-b328-2789f0c270d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:18.680036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-71342575-3499-40d0-b328-2789f0c270d5 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:18.681221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-71342575-3499-40d0-b328-2789f0c270d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:18.689671 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1014/2016] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:18.693918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5cd77-7148-435b-a40f-a3deb84a6da4 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:18.704001 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5cd77-7148-435b-a40f-a3deb84a6da4 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:18.708493 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f51de-a8f6-4686-b0db-ebdcd5c00204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:18.709088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f51de-a8f6-4686-b0db-ebdcd5c00204 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:18.709364 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f51de-a8f6-4686-b0db-ebdcd5c00204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:18.709631 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f51de-a8f6-4686-b0db-ebdcd5c00204 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:18.710268 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f51de-a8f6-4686-b0db-ebdcd5c00204 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:23:18.710566 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f51de-a8f6-4686-b0db-ebdcd5c00204 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:18.712212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f51de-a8f6-4686-b0db-ebdcd5c00204 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:18.724296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1003/2017] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:23:18.731874 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71342575-3499-40d0-b328-2789f0c270d5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:18.741319 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71342575-3499-40d0-b328-2789f0c270d5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:18.746596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48cea4a-f561-4675-a341-dbefed023d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:18.747202 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48cea4a-f561-4675-a341-dbefed023d48 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:18.747487 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48cea4a-f561-4675-a341-dbefed023d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:18.747791 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48cea4a-f561-4675-a341-dbefed023d48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:18.748418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48cea4a-f561-4675-a341-dbefed023d48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:18.750305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48cea4a-f561-4675-a341-dbefed023d48 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:18.778236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1015/2018] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:23:18 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:18.782337 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f51de-a8f6-4686-b0db-ebdcd5c00204 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:18.793309 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f51de-a8f6-4686-b0db-ebdcd5c00204 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:18.818181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:18.851413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cc949-8107-4f7a-af9e-34f4152bd260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:18.852042 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cc949-8107-4f7a-af9e-34f4152bd260 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:18.852319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cc949-8107-4f7a-af9e-34f4152bd260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:18.852588 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cc949-8107-4f7a-af9e-34f4152bd260 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:18.853204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799cc949-8107-4f7a-af9e-34f4152bd260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:18.855626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799cc949-8107-4f7a-af9e-34f4152bd260 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:18.875936 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1004/2019] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:23:18 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:18.879993 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48cea4a-f561-4675-a341-dbefed023d48 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:18.890694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48cea4a-f561-4675-a341-dbefed023d48 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:18.895307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64449ff2-4762-4b18-9a96-cb3ca89f79bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:18.895960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64449ff2-4762-4b18-9a96-cb3ca89f79bc admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:18.896243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64449ff2-4762-4b18-9a96-cb3ca89f79bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:18.896515 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64449ff2-4762-4b18-9a96-cb3ca89f79bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:18.897148 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64449ff2-4762-4b18-9a96-cb3ca89f79bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:18.898730 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64449ff2-4762-4b18-9a96-cb3ca89f79bc admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:18.908504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1016/2020] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:18.912732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cc949-8107-4f7a-af9e-34f4152bd260 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:18.923156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cc949-8107-4f7a-af9e-34f4152bd260 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:18.927807 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37357a0a-eebc-47f8-9fcc-98b595fd4a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:18.928345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37357a0a-eebc-47f8-9fcc-98b595fd4a73 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:18.928624 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37357a0a-eebc-47f8-9fcc-98b595fd4a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:18.928895 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37357a0a-eebc-47f8-9fcc-98b595fd4a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:18.929531 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37357a0a-eebc-47f8-9fcc-98b595fd4a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:18.931129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37357a0a-eebc-47f8-9fcc-98b595fd4a73 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:18.940862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1005/2021] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:18.944709 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64449ff2-4762-4b18-9a96-cb3ca89f79bc admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:18.954923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64449ff2-4762-4b18-9a96-cb3ca89f79bc admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:18.959481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30201120-e343-4fde-b25a-0636e2545135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:18.960147 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30201120-e343-4fde-b25a-0636e2545135 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:18.960430 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30201120-e343-4fde-b25a-0636e2545135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:18.960706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30201120-e343-4fde-b25a-0636e2545135 admin admin] PATH_INFO: `/v3/projects/515a7064170b4dc2aa20063e794deeae/users/6a8362868ce0431fb69edfa399ecc3a3/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:18.975476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30201120-e343-4fde-b25a-0636e2545135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=515a7064170b4dc2aa20063e794deeae, user_id=6a8362868ce0431fb69edfa399ecc3a3, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:18.978147 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30201120-e343-4fde-b25a-0636e2545135 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:18.990895 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1017/2022] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:18 2022] PUT /identity/v3/projects/515a7064170b4dc2aa20063e794deeae/users/6a8362868ce0431fb69edfa399ecc3a3/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:19.001653 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52122f-28d7-497e-aa1a-692edab15419 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:19.002112 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52122f-28d7-497e-aa1a-692edab15419 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:19.002224 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52122f-28d7-497e-aa1a-692edab15419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:19.031908 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6d52122f-28d7-497e-aa1a-692edab15419 None None] MFA Rules not processed for user `6a8362868ce0431fb69edfa399ecc3a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:19.033182 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6d52122f-28d7-497e-aa1a-692edab15419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:19.098593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1006/2023] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:19.109423 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30201120-e343-4fde-b25a-0636e2545135 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:19.111114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-30201120-e343-4fde-b25a-0636e2545135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:19.167761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30201120-e343-4fde-b25a-0636e2545135 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:19.171660 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015fb70d-fb16-450d-916c-fca2e327e257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:19.172281 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fb70d-fb16-450d-916c-fca2e327e257 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:19.172630 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fb70d-fb16-450d-916c-fca2e327e257 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:19.172913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fb70d-fb16-450d-916c-fca2e327e257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:19.182399 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015fb70d-fb16-450d-916c-fca2e327e257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:19.183841 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015fb70d-fb16-450d-916c-fca2e327e257 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:19.193265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1018/2024] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:29.097306 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:29.190487 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:32.954487 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52122f-28d7-497e-aa1a-692edab15419 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:32.956659 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6d52122f-28d7-497e-aa1a-692edab15419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:33.029524 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52122f-28d7-497e-aa1a-692edab15419 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:33.035006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41026a9f-b46d-4437-a273-ed4e3f913fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:33.035672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41026a9f-b46d-4437-a273-ed4e3f913fdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:33.035950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41026a9f-b46d-4437-a273-ed4e3f913fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:33.036219 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41026a9f-b46d-4437-a273-ed4e3f913fdf admin admin] PATH_INFO: `/v3/users/837394ab068e4f62aa1e725a59f083d0` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:33.037325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41026a9f-b46d-4437-a273-ed4e3f913fdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=837394ab068e4f62aa1e725a59f083d0)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:33.039237 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41026a9f-b46d-4437-a273-ed4e3f913fdf admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:33.096778 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-41026a9f-b46d-4437-a273-ed4e3f913fdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '837394ab068e4f62aa1e725a59f083d0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:33.105338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-41026a9f-b46d-4437-a273-ed4e3f913fdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '837394ab068e4f62aa1e725a59f083d0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:33.106222 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-41026a9f-b46d-4437-a273-ed4e3f913fdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '837394ab068e4f62aa1e725a59f083d0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:33.106980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-41026a9f-b46d-4437-a273-ed4e3f913fdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '837394ab068e4f62aa1e725a59f083d0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:33.116067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-41026a9f-b46d-4437-a273-ed4e3f913fdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '837394ab068e4f62aa1e725a59f083d0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:33.135026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1007/2025] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:23:32 2022] DELETE /identity/v3/users/837394ab068e4f62aa1e725a59f083d0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:33.142985 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b98c1e-68f4-4ed7-93ea-6ddaab974be5 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:33.143442 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b98c1e-68f4-4ed7-93ea-6ddaab974be5 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:33.143785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b98c1e-68f4-4ed7-93ea-6ddaab974be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:33.167410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f7b98c1e-68f4-4ed7-93ea-6ddaab974be5 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:33.168875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f7b98c1e-68f4-4ed7-93ea-6ddaab974be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:33.237532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1019/2026] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:33.244397 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41026a9f-b46d-4437-a273-ed4e3f913fdf admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:33.245890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-41026a9f-b46d-4437-a273-ed4e3f913fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:33.309280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41026a9f-b46d-4437-a273-ed4e3f913fdf admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:33.313496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e73ea-c068-46ee-af04-76317d8ab2c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:33.314264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e73ea-c068-46ee-af04-76317d8ab2c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:33.314716 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e73ea-c068-46ee-af04-76317d8ab2c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:33.315886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e73ea-c068-46ee-af04-76317d8ab2c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:33.328530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e73ea-c068-46ee-af04-76317d8ab2c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:33.331158 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e73ea-c068-46ee-af04-76317d8ab2c9 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:33.344515 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1008/2027] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:33.662859 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b98c1e-68f4-4ed7-93ea-6ddaab974be5 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:33.664433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f7b98c1e-68f4-4ed7-93ea-6ddaab974be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:33.687572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b98c1e-68f4-4ed7-93ea-6ddaab974be5 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:33.704784 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1f0cf-71d4-4ba3-bf40-e1777bafc6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:33.706138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1f0cf-71d4-4ba3-bf40-e1777bafc6fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:33.706430 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1f0cf-71d4-4ba3-bf40-e1777bafc6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:33.707330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1f0cf-71d4-4ba3-bf40-e1777bafc6fa admin admin] PATH_INFO: `/v3/projects/f4fedbcdda2c4b6dac6268c093008642` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:33.711196 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1f0cf-71d4-4ba3-bf40-e1777bafc6fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4fedbcdda2c4b6dac6268c093008642)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:33.715106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1f0cf-71d4-4ba3-bf40-e1777bafc6fa admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:33.771943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c6f1f0cf-71d4-4ba3-bf40-e1777bafc6fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4fedbcdda2c4b6dac6268c093008642', 'request_id': 'req-c6f1f0cf-71d4-4ba3-bf40-e1777bafc6fa'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:33.782005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c6f1f0cf-71d4-4ba3-bf40-e1777bafc6fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4fedbcdda2c4b6dac6268c093008642', 'request_id': 'req-c6f1f0cf-71d4-4ba3-bf40-e1777bafc6fa'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:33.791930 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c6f1f0cf-71d4-4ba3-bf40-e1777bafc6fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4fedbcdda2c4b6dac6268c093008642', 'request_id': 'req-c6f1f0cf-71d4-4ba3-bf40-e1777bafc6fa'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:33.803794 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c6f1f0cf-71d4-4ba3-bf40-e1777bafc6fa admin admin] The token cache is being invalidate because project f4fedbcdda2c4b6dac6268c093008642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:23:33.804072 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c6f1f0cf-71d4-4ba3-bf40-e1777bafc6fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:33.805104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-c6f1f0cf-71d4-4ba3-bf40-e1777bafc6fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:33.807291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1020/2028] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:23:33 2022] DELETE /identity/v3/projects/f4fedbcdda2c4b6dac6268c093008642 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:33.812856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28a3e6-b27b-4d03-8941-c3a432794610 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:33.813167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28a3e6-b27b-4d03-8941-c3a432794610 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:33.813488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28a3e6-b27b-4d03-8941-c3a432794610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:33.833818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8f28a3e6-b27b-4d03-8941-c3a432794610 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:33.835004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8f28a3e6-b27b-4d03-8941-c3a432794610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:33.854580 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:33.905346 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1009/2029] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:33.913247 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97da494-f590-4d67-a009-7445e63df32e None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:33.913533 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97da494-f590-4d67-a009-7445e63df32e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:33.913806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97da494-f590-4d67-a009-7445e63df32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:33.933552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d97da494-f590-4d67-a009-7445e63df32e None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:33.934754 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d97da494-f590-4d67-a009-7445e63df32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:33.942841 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1021/2030] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:33.947743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28a3e6-b27b-4d03-8941-c3a432794610 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:33.957093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28a3e6-b27b-4d03-8941-c3a432794610 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:33.962654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc9954-6f7f-4cc6-b6e2-c09e5fa9ee33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:33.963419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc9954-6f7f-4cc6-b6e2-c09e5fa9ee33 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:33.963826 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc9954-6f7f-4cc6-b6e2-c09e5fa9ee33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:33.964106 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc9954-6f7f-4cc6-b6e2-c09e5fa9ee33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:33.964834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc9954-6f7f-4cc6-b6e2-c09e5fa9ee33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:23:33.965083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc9954-6f7f-4cc6-b6e2-c09e5fa9ee33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:33.966690 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc9954-6f7f-4cc6-b6e2-c09e5fa9ee33 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:33.979827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1010/2031] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:23:33.984342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97da494-f590-4d67-a009-7445e63df32e None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:33.994806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97da494-f590-4d67-a009-7445e63df32e None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:34.000801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95450288-d93a-454f-aa3e-269540b80832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:34.001442 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95450288-d93a-454f-aa3e-269540b80832 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:34.002239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95450288-d93a-454f-aa3e-269540b80832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:34.002239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95450288-d93a-454f-aa3e-269540b80832 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:34.002633 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95450288-d93a-454f-aa3e-269540b80832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:34.005087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95450288-d93a-454f-aa3e-269540b80832 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:34.047267 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1022/2032] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:23:33 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:34.049134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc9954-6f7f-4cc6-b6e2-c09e5fa9ee33 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:34.064212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc9954-6f7f-4cc6-b6e2-c09e5fa9ee33 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:34.123133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700829c5-1e01-4b7f-b3fa-919c608a61ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:34.123818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700829c5-1e01-4b7f-b3fa-919c608a61ff admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:34.124104 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700829c5-1e01-4b7f-b3fa-919c608a61ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:34.124377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700829c5-1e01-4b7f-b3fa-919c608a61ff admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:34.125008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700829c5-1e01-4b7f-b3fa-919c608a61ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:34.127003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700829c5-1e01-4b7f-b3fa-919c608a61ff admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:34.149672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1011/2033] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:23:34 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:34.153104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95450288-d93a-454f-aa3e-269540b80832 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:34.163441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95450288-d93a-454f-aa3e-269540b80832 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:34.168067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d8b7a-1a3a-4674-b815-c284b231ba0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:34.168656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d8b7a-1a3a-4674-b815-c284b231ba0e admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:34.168931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d8b7a-1a3a-4674-b815-c284b231ba0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:34.169200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d8b7a-1a3a-4674-b815-c284b231ba0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:34.169831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d8b7a-1a3a-4674-b815-c284b231ba0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:34.171398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d8b7a-1a3a-4674-b815-c284b231ba0e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:34.182513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1023/2034] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:34.186260 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700829c5-1e01-4b7f-b3fa-919c608a61ff admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:34.195891 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700829c5-1e01-4b7f-b3fa-919c608a61ff admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:34.200961 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e59cb-0848-4917-88aa-f4f8dd119e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:34.201574 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e59cb-0848-4917-88aa-f4f8dd119e7d admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:34.201853 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e59cb-0848-4917-88aa-f4f8dd119e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:34.202119 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e59cb-0848-4917-88aa-f4f8dd119e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:34.202750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e59cb-0848-4917-88aa-f4f8dd119e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:34.204361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e59cb-0848-4917-88aa-f4f8dd119e7d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:34.214585 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1012/2035] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:34.218451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d8b7a-1a3a-4674-b815-c284b231ba0e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:34.228417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d8b7a-1a3a-4674-b815-c284b231ba0e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:34.233036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b714430-fd63-4e70-8025-e1be9523f573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:34.233677 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b714430-fd63-4e70-8025-e1be9523f573 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:34.233958 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b714430-fd63-4e70-8025-e1be9523f573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:34.234230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b714430-fd63-4e70-8025-e1be9523f573 admin admin] PATH_INFO: `/v3/projects/d785e123e16648908d42c04a0130dc0e/users/1cfa4ce1c25647269b81a514e3f2a271/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:34.249363 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b714430-fd63-4e70-8025-e1be9523f573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d785e123e16648908d42c04a0130dc0e, user_id=1cfa4ce1c25647269b81a514e3f2a271, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:34.252113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b714430-fd63-4e70-8025-e1be9523f573 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:34.265277 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1024/2036] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:34 2022] PUT /identity/v3/projects/d785e123e16648908d42c04a0130dc0e/users/1cfa4ce1c25647269b81a514e3f2a271/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:34.275445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a578e5e-dd48-4540-a938-523ef963896f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:34.275771 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a578e5e-dd48-4540-a938-523ef963896f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:34.276044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a578e5e-dd48-4540-a938-523ef963896f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:34.303880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-4a578e5e-dd48-4540-a938-523ef963896f None None] MFA Rules not processed for user `1cfa4ce1c25647269b81a514e3f2a271`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:34.305086 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4a578e5e-dd48-4540-a938-523ef963896f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:34.367150 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1013/2037] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:34.378427 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b714430-fd63-4e70-8025-e1be9523f573 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:34.380426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8b714430-fd63-4e70-8025-e1be9523f573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:34.405721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:34.444492 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b714430-fd63-4e70-8025-e1be9523f573 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:34.448229 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2489e-d1d4-448d-823d-854e87117588 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:34.448822 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2489e-d1d4-448d-823d-854e87117588 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:34.449097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2489e-d1d4-448d-823d-854e87117588 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:34.449366 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2489e-d1d4-448d-823d-854e87117588 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:34.457306 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2489e-d1d4-448d-823d-854e87117588 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:34.458814 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2489e-d1d4-448d-823d-854e87117588 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:34.468248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1025/2038] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:44.364370 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:44.468052 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=17 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:52.093741 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a578e5e-dd48-4540-a938-523ef963896f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:52.096971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4a578e5e-dd48-4540-a938-523ef963896f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:52.205856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a578e5e-dd48-4540-a938-523ef963896f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:52.211496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a1166-223f-4bc3-8b1d-43be5b340ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:52.212228 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a1166-223f-4bc3-8b1d-43be5b340ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:52.212508 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a1166-223f-4bc3-8b1d-43be5b340ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:52.212914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a1166-223f-4bc3-8b1d-43be5b340ea0 admin admin] PATH_INFO: `/v3/users/6a8362868ce0431fb69edfa399ecc3a3` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:52.214217 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a1166-223f-4bc3-8b1d-43be5b340ea0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a8362868ce0431fb69edfa399ecc3a3)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:52.216339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a1166-223f-4bc3-8b1d-43be5b340ea0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:52.310316 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-877a1166-223f-4bc3-8b1d-43be5b340ea0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a8362868ce0431fb69edfa399ecc3a3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:52.318409 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-877a1166-223f-4bc3-8b1d-43be5b340ea0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a8362868ce0431fb69edfa399ecc3a3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:52.319299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-877a1166-223f-4bc3-8b1d-43be5b340ea0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a8362868ce0431fb69edfa399ecc3a3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:52.321217 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-877a1166-223f-4bc3-8b1d-43be5b340ea0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a8362868ce0431fb69edfa399ecc3a3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:52.327348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2489e-d1d4-448d-823d-854e87117588 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:52.329530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-87c2489e-d1d4-448d-823d-854e87117588 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:52.333714 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-877a1166-223f-4bc3-8b1d-43be5b340ea0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a8362868ce0431fb69edfa399ecc3a3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:52.352177 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2489e-d1d4-448d-823d-854e87117588 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:52.356320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1014/2039] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:23:52 2022] DELETE /identity/v3/users/6a8362868ce0431fb69edfa399ecc3a3 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:52.359197 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99e992-885a-4dde-9b8c-07ca91697269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:52.359947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99e992-885a-4dde-9b8c-07ca91697269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:52.360484 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b443fb-f6cf-4e81-9859-6f4fb8377f72 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:52.360596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99e992-885a-4dde-9b8c-07ca91697269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:52.360672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b443fb-f6cf-4e81-9859-6f4fb8377f72 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:52.360762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99e992-885a-4dde-9b8c-07ca91697269 admin admin] PATH_INFO: `/v3/users/15ba03725c8449fb9c9eb053ea6e4606` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:52.360838 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b443fb-f6cf-4e81-9859-6f4fb8377f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:52.362269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99e992-885a-4dde-9b8c-07ca91697269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15ba03725c8449fb9c9eb053ea6e4606)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:52.364298 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99e992-885a-4dde-9b8c-07ca91697269 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:52.383961 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-21b443fb-f6cf-4e81-9859-6f4fb8377f72 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:52.385222 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-21b443fb-f6cf-4e81-9859-6f4fb8377f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:52.419157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5f99e992-885a-4dde-9b8c-07ca91697269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15ba03725c8449fb9c9eb053ea6e4606'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:52.428349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5f99e992-885a-4dde-9b8c-07ca91697269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15ba03725c8449fb9c9eb053ea6e4606'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:52.429156 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5f99e992-885a-4dde-9b8c-07ca91697269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15ba03725c8449fb9c9eb053ea6e4606'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:52.430570 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5f99e992-885a-4dde-9b8c-07ca91697269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15ba03725c8449fb9c9eb053ea6e4606'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:52.439957 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5f99e992-885a-4dde-9b8c-07ca91697269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15ba03725c8449fb9c9eb053ea6e4606'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:52.461813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1015/2040] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:52.463579 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1026/2041] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:23:52 2022] DELETE /identity/v3/users/15ba03725c8449fb9c9eb053ea6e4606 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:23:52.469165 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7cc39-a07c-424f-93ab-46d867ff2d82 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:52.469422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7cc39-a07c-424f-93ab-46d867ff2d82 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:52.469693 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7cc39-a07c-424f-93ab-46d867ff2d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:52.472440 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99e992-885a-4dde-9b8c-07ca91697269 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:52.474150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5f99e992-885a-4dde-9b8c-07ca91697269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:52.502926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-52d7cc39-a07c-424f-93ab-46d867ff2d82 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:52.515537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-52d7cc39-a07c-424f-93ab-46d867ff2d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:52.526669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:52.553451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99e992-885a-4dde-9b8c-07ca91697269 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:52.557703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cfa98-1e7e-46b8-a86b-c0bf13ed583d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:52.558394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cfa98-1e7e-46b8-a86b-c0bf13ed583d service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:52.558750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cfa98-1e7e-46b8-a86b-c0bf13ed583d service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:52.559097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cfa98-1e7e-46b8-a86b-c0bf13ed583d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:52.586036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1016/2042] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:52.598215 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7cc39-a07c-424f-93ab-46d867ff2d82 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:52.599985 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-52d7cc39-a07c-424f-93ab-46d867ff2d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:52.613865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7cc39-a07c-424f-93ab-46d867ff2d82 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:52.618967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150ec1d-0d18-4866-b419-79913529961c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:52.619608 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ec1d-0d18-4866-b419-79913529961c service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:52.619890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ec1d-0d18-4866-b419-79913529961c service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:52.620256 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ec1d-0d18-4866-b419-79913529961c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:52.628538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cfa98-1e7e-46b8-a86b-c0bf13ed583d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:52.629451 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150ec1d-0d18-4866-b419-79913529961c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:52.629980 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cfa98-1e7e-46b8-a86b-c0bf13ed583d service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:52.631144 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150ec1d-0d18-4866-b419-79913529961c service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:52.641007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1027/2043] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:52.642503 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1017/2044] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:23:53.081243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cfa98-1e7e-46b8-a86b-c0bf13ed583d service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:53.083032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6a1cfa98-1e7e-46b8-a86b-c0bf13ed583d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:53.092168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150ec1d-0d18-4866-b419-79913529961c service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:53.093528 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3150ec1d-0d18-4866-b419-79913529961c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:53.103882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cfa98-1e7e-46b8-a86b-c0bf13ed583d service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:53.107324 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150ec1d-0d18-4866-b419-79913529961c service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:53.109534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0db13-1153-46b8-8ce2-b89996c6bf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:53.110170 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0db13-1153-46b8-8ce2-b89996c6bf92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:53.110501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0db13-1153-46b8-8ce2-b89996c6bf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:53.110781 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0db13-1153-46b8-8ce2-b89996c6bf92 admin admin] PATH_INFO: `/v3/projects/515a7064170b4dc2aa20063e794deeae` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:53.112231 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0db13-1153-46b8-8ce2-b89996c6bf92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=515a7064170b4dc2aa20063e794deeae)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:53.114789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0db13-1153-46b8-8ce2-b89996c6bf92 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:53.116563 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:53.117555 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:53.118110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:53.118595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a admin admin] PATH_INFO: `/v3/projects/d7ec442aa479492783daf603f9a284ce` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:53.120029 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7ec442aa479492783daf603f9a284ce)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:53.122135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:53.137502 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:53.195210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7ec442aa479492783daf603f9a284ce', 'request_id': 'req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:53.205779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7ec442aa479492783daf603f9a284ce', 'request_id': 'req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:53.209317 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b4e0db13-1153-46b8-8ce2-b89996c6bf92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '515a7064170b4dc2aa20063e794deeae', 'request_id': 'req-b4e0db13-1153-46b8-8ce2-b89996c6bf92'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:53.215110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7ec442aa479492783daf603f9a284ce', 'request_id': 'req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:53.218441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b4e0db13-1153-46b8-8ce2-b89996c6bf92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '515a7064170b4dc2aa20063e794deeae', 'request_id': 'req-b4e0db13-1153-46b8-8ce2-b89996c6bf92'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:53.226305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b4e0db13-1153-46b8-8ce2-b89996c6bf92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '515a7064170b4dc2aa20063e794deeae', 'request_id': 'req-b4e0db13-1153-46b8-8ce2-b89996c6bf92'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:53.227197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a admin admin] The token cache is being invalidate because project d7ec442aa479492783daf603f9a284ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:23:53.227601 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:53.228264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:53.229786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1018/2045] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:23:53 2022] DELETE /identity/v3/projects/d7ec442aa479492783daf603f9a284ce => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:53.233729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:53.235128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:53.238490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b4e0db13-1153-46b8-8ce2-b89996c6bf92 admin admin] The token cache is being invalidate because project 515a7064170b4dc2aa20063e794deeae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:23:53.238855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b4e0db13-1153-46b8-8ce2-b89996c6bf92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:53.239655 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b4e0db13-1153-46b8-8ce2-b89996c6bf92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:53.241173 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1028/2046] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:23:53 2022] DELETE /identity/v3/projects/515a7064170b4dc2aa20063e794deeae => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:53.245963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927bdb3-0f73-41b9-857f-208f5be0a35a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:53.246829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927bdb3-0f73-41b9-857f-208f5be0a35a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:53.246829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927bdb3-0f73-41b9-857f-208f5be0a35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:53.267957 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d927bdb3-0f73-41b9-857f-208f5be0a35a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:53.269263 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d927bdb3-0f73-41b9-857f-208f5be0a35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:53.317305 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010eaecf-8343-4ff4-89f7-f9bd46d4cd8a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:53.339008 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1029/2047] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:23:53.347028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7a808-4349-421b-801a-d729f6f49665 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:53.347417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7a808-4349-421b-801a-d729f6f49665 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:53.347819 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7a808-4349-421b-801a-d729f6f49665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:53.389132 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-bac7a808-4349-421b-801a-d729f6f49665 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:53.390578 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bac7a808-4349-421b-801a-d729f6f49665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:53.398887 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:53.403435 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1030/2048] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:53.404572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7a808-4349-421b-801a-d729f6f49665 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:53.428592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7a808-4349-421b-801a-d729f6f49665 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:53.434328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2afe73-8a02-43e1-9b18-ac7b58a5d287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:53.435070 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2afe73-8a02-43e1-9b18-ac7b58a5d287 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:53.435492 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2afe73-8a02-43e1-9b18-ac7b58a5d287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:53.436105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2afe73-8a02-43e1-9b18-ac7b58a5d287 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:53.436839 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2afe73-8a02-43e1-9b18-ac7b58a5d287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:23:53.437218 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2afe73-8a02-43e1-9b18-ac7b58a5d287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:53.437510 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d30b0a-27b6-4554-a98b-957305b9a362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:53.438516 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d30b0a-27b6-4554-a98b-957305b9a362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:53.438649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d30b0a-27b6-4554-a98b-957305b9a362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:53.438967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d30b0a-27b6-4554-a98b-957305b9a362 admin admin] PATH_INFO: `/v3/users/5af0791c13bf4716871968a725c42de2` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:53.439589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2afe73-8a02-43e1-9b18-ac7b58a5d287 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:53.443744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d30b0a-27b6-4554-a98b-957305b9a362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5af0791c13bf4716871968a725c42de2)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:53.443744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d30b0a-27b6-4554-a98b-957305b9a362 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:53.459607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1031/2049] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:23:53.464215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2afe73-8a02-43e1-9b18-ac7b58a5d287 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:53.477598 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2afe73-8a02-43e1-9b18-ac7b58a5d287 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:53.487638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004850fd-4882-482e-a44d-f1767b92258c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:53.488250 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004850fd-4882-482e-a44d-f1767b92258c admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:53.488528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004850fd-4882-482e-a44d-f1767b92258c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:53.488800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004850fd-4882-482e-a44d-f1767b92258c admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:53.489435 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004850fd-4882-482e-a44d-f1767b92258c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:53.491325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004850fd-4882-482e-a44d-f1767b92258c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:53.526550 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-75d30b0a-27b6-4554-a98b-957305b9a362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5af0791c13bf4716871968a725c42de2'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:53.529202 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1032/2050] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:23:53 2022] POST /identity/v3/projects => generated 391 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:23:53.537320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-75d30b0a-27b6-4554-a98b-957305b9a362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5af0791c13bf4716871968a725c42de2'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:53.538330 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-75d30b0a-27b6-4554-a98b-957305b9a362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5af0791c13bf4716871968a725c42de2'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:53.539351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-75d30b0a-27b6-4554-a98b-957305b9a362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5af0791c13bf4716871968a725c42de2'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:53.539852 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004850fd-4882-482e-a44d-f1767b92258c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:53.546827 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-004850fd-4882-482e-a44d-f1767b92258c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:53.549290 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-75d30b0a-27b6-4554-a98b-957305b9a362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5af0791c13bf4716871968a725c42de2'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:53.590848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1019/2051] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:23:53 2022] DELETE /identity/v3/users/5af0791c13bf4716871968a725c42de2 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:23:53.638371 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004850fd-4882-482e-a44d-f1767b92258c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:53.649074 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:53.721165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28a584-824b-46b2-b305-a3719ec2f878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:53.721931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28a584-824b-46b2-b305-a3719ec2f878 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:53.722281 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28a584-824b-46b2-b305-a3719ec2f878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:53.723090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28a584-824b-46b2-b305-a3719ec2f878 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:53.724190 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28a584-824b-46b2-b305-a3719ec2f878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:53.727196 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28a584-824b-46b2-b305-a3719ec2f878 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:53.761540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1033/2052] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:23:53 2022] POST /identity/v3/users => generated 296 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:53.765714 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d30b0a-27b6-4554-a98b-957305b9a362 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:53.776535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d30b0a-27b6-4554-a98b-957305b9a362 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:53.781430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e55827-19e7-4319-8caf-737f039b77cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:53.781858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e55827-19e7-4319-8caf-737f039b77cc admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:53.782142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e55827-19e7-4319-8caf-737f039b77cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:53.782430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e55827-19e7-4319-8caf-737f039b77cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:53.783063 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e55827-19e7-4319-8caf-737f039b77cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:53.784962 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e55827-19e7-4319-8caf-737f039b77cc admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:53.795368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1020/2053] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:53.800711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28a584-824b-46b2-b305-a3719ec2f878 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:53.811787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28a584-824b-46b2-b305-a3719ec2f878 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:53.816499 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74619b53-c66c-44fd-be9f-b76fb0366c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:53.817111 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74619b53-c66c-44fd-be9f-b76fb0366c33 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:53.817393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74619b53-c66c-44fd-be9f-b76fb0366c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:53.817665 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74619b53-c66c-44fd-be9f-b76fb0366c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:53.818342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74619b53-c66c-44fd-be9f-b76fb0366c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:53.820539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74619b53-c66c-44fd-be9f-b76fb0366c33 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:53.831184 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1034/2054] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:53.835113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e55827-19e7-4319-8caf-737f039b77cc admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:53.845518 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e55827-19e7-4319-8caf-737f039b77cc admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:53.850425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3b97d-40bc-4256-8cb4-f4ce606b7c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:53.851084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3b97d-40bc-4256-8cb4-f4ce606b7c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:53.851418 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3b97d-40bc-4256-8cb4-f4ce606b7c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:53.851731 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3b97d-40bc-4256-8cb4-f4ce606b7c81 admin admin] PATH_INFO: `/v3/projects/f378e738b9c144799bf5f4b875048455/users/9bdc859ae3e44327b52d1dca32279c05/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:53.868604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3b97d-40bc-4256-8cb4-f4ce606b7c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f378e738b9c144799bf5f4b875048455, user_id=9bdc859ae3e44327b52d1dca32279c05, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:53.871248 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3b97d-40bc-4256-8cb4-f4ce606b7c81 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:53.886671 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1021/2055] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:53 2022] PUT /identity/v3/projects/f378e738b9c144799bf5f4b875048455/users/9bdc859ae3e44327b52d1dca32279c05/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:53.898533 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d7b6f-54ad-42b2-9f74-643c01a49e95 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:53.898960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d7b6f-54ad-42b2-9f74-643c01a49e95 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:53.899417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d7b6f-54ad-42b2-9f74-643c01a49e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:53.928665 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-911d7b6f-54ad-42b2-9f74-643c01a49e95 None None] MFA Rules not processed for user `9bdc859ae3e44327b52d1dca32279c05`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:53.930914 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-911d7b6f-54ad-42b2-9f74-643c01a49e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:54.007711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1035/2056] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:23:54.021981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3b97d-40bc-4256-8cb4-f4ce606b7c81 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:54.024382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ced3b97d-40bc-4256-8cb4-f4ce606b7c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:54.090315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3b97d-40bc-4256-8cb4-f4ce606b7c81 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:54.094309 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c4053-fbaf-42c4-b08d-5813b001f908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:54.095294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c4053-fbaf-42c4-b08d-5813b001f908 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:54.095840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c4053-fbaf-42c4-b08d-5813b001f908 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:54.096179 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c4053-fbaf-42c4-b08d-5813b001f908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:54.100411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d7b6f-54ad-42b2-9f74-643c01a49e95 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:54.102415 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-911d7b6f-54ad-42b2-9f74-643c01a49e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:54.106447 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c4053-fbaf-42c4-b08d-5813b001f908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:54.108064 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c4053-fbaf-42c4-b08d-5813b001f908 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:54.118027 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1022/2057] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:54.168090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d7b6f-54ad-42b2-9f74-643c01a49e95 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:54.174174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8426b81-0f0a-4e10-9b9b-a143576f3baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:54.174830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8426b81-0f0a-4e10-9b9b-a143576f3baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:54.175096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8426b81-0f0a-4e10-9b9b-a143576f3baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:54.175371 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8426b81-0f0a-4e10-9b9b-a143576f3baa admin admin] PATH_INFO: `/v3/users/5881c80a14fa4c19aa06ee172f179ea4` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:54.176565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8426b81-0f0a-4e10-9b9b-a143576f3baa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5881c80a14fa4c19aa06ee172f179ea4)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:54.178508 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8426b81-0f0a-4e10-9b9b-a143576f3baa admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:54.189159 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c4053-fbaf-42c4-b08d-5813b001f908 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:54.190829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-075c4053-fbaf-42c4-b08d-5813b001f908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:54.208722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:54.209668 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c4053-fbaf-42c4-b08d-5813b001f908 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:54.221195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a7530-ec75-4a97-acff-881e019888e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:54.222615 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a7530-ec75-4a97-acff-881e019888e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:54.222969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a7530-ec75-4a97-acff-881e019888e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:54.223198 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a7530-ec75-4a97-acff-881e019888e5 admin admin] PATH_INFO: `/v3/users/1cfa4ce1c25647269b81a514e3f2a271` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:54.226963 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a7530-ec75-4a97-acff-881e019888e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cfa4ce1c25647269b81a514e3f2a271)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:54.232514 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a7530-ec75-4a97-acff-881e019888e5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:54.252656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a8426b81-0f0a-4e10-9b9b-a143576f3baa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5881c80a14fa4c19aa06ee172f179ea4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:54.268683 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a8426b81-0f0a-4e10-9b9b-a143576f3baa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5881c80a14fa4c19aa06ee172f179ea4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:54.269559 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a8426b81-0f0a-4e10-9b9b-a143576f3baa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5881c80a14fa4c19aa06ee172f179ea4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:54.272075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a8426b81-0f0a-4e10-9b9b-a143576f3baa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5881c80a14fa4c19aa06ee172f179ea4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:54.278997 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:54.285462 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a8426b81-0f0a-4e10-9b9b-a143576f3baa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5881c80a14fa4c19aa06ee172f179ea4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:54.325797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-224a7530-ec75-4a97-acff-881e019888e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cfa4ce1c25647269b81a514e3f2a271'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:54.336590 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-224a7530-ec75-4a97-acff-881e019888e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cfa4ce1c25647269b81a514e3f2a271'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:54.338438 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-224a7530-ec75-4a97-acff-881e019888e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cfa4ce1c25647269b81a514e3f2a271'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:54.339659 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-224a7530-ec75-4a97-acff-881e019888e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cfa4ce1c25647269b81a514e3f2a271'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:54.347873 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1036/2058] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:23:54 2022] DELETE /identity/v3/users/5881c80a14fa4c19aa06ee172f179ea4 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:54.350771 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8426b81-0f0a-4e10-9b9b-a143576f3baa admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:54.352728 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-224a7530-ec75-4a97-acff-881e019888e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cfa4ce1c25647269b81a514e3f2a271'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:54.354782 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-a8426b81-0f0a-4e10-9b9b-a143576f3baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:54.379976 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1023/2059] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:23:54 2022] DELETE /identity/v3/users/1cfa4ce1c25647269b81a514e3f2a271 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:23:54.385659 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4075e-af59-4944-9592-47553b84361f None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:54.386072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4075e-af59-4944-9592-47553b84361f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:54.386516 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4075e-af59-4944-9592-47553b84361f None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:54.388082 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1024/2060] 10.222.0.33 () {54 vars in 921 bytes} [Thu Oct 6 12:23:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:23:54.389901 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398de36-0c2a-4345-9a5c-a1469b6f059c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:54.390295 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398de36-0c2a-4345-9a5c-a1469b6f059c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:54.391252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398de36-0c2a-4345-9a5c-a1469b6f059c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:54.424967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-0398de36-0c2a-4345-9a5c-a1469b6f059c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:54.433937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0398de36-0c2a-4345-9a5c-a1469b6f059c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:54.461476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8426b81-0f0a-4e10-9b9b-a143576f3baa admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:54.539483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1025/2061] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:54.549996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae85e2-b02b-47eb-b3f7-ded5ce6e22bb None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:54.550306 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae85e2-b02b-47eb-b3f7-ded5ce6e22bb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:54.550588 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae85e2-b02b-47eb-b3f7-ded5ce6e22bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:54.565029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0bd3f-2c84-4b37-b780-028396f175ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:54.566328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0bd3f-2c84-4b37-b780-028396f175ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:54.566684 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0bd3f-2c84-4b37-b780-028396f175ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:54.567029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0bd3f-2c84-4b37-b780-028396f175ce admin admin] PATH_INFO: `/v3/projects/41807e498b3947aca80280634840ef6b` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:54.568810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0bd3f-2c84-4b37-b780-028396f175ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=41807e498b3947aca80280634840ef6b)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:54.573131 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0bd3f-2c84-4b37-b780-028396f175ce admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:54.587191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-afae85e2-b02b-47eb-b3f7-ded5ce6e22bb None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:54.589623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-afae85e2-b02b-47eb-b3f7-ded5ce6e22bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:54.611690 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1026/2062] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:54.612624 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865915aa-dcf3-4332-b49a-fb9e950f41e5 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:54.612915 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865915aa-dcf3-4332-b49a-fb9e950f41e5 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:54.613871 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865915aa-dcf3-4332-b49a-fb9e950f41e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:54.682122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-865915aa-dcf3-4332-b49a-fb9e950f41e5 None None] MFA Rules not processed for user `39d67f5edf4b441ebaa2bb42e74b2cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:54.684166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-865915aa-dcf3-4332-b49a-fb9e950f41e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:54.699354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:54.701965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6ad0bd3f-2c84-4b37-b780-028396f175ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41807e498b3947aca80280634840ef6b', 'request_id': 'req-6ad0bd3f-2c84-4b37-b780-028396f175ce'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:54.722104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6ad0bd3f-2c84-4b37-b780-028396f175ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41807e498b3947aca80280634840ef6b', 'request_id': 'req-6ad0bd3f-2c84-4b37-b780-028396f175ce'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:54.740592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6ad0bd3f-2c84-4b37-b780-028396f175ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41807e498b3947aca80280634840ef6b', 'request_id': 'req-6ad0bd3f-2c84-4b37-b780-028396f175ce'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:54.788512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6ad0bd3f-2c84-4b37-b780-028396f175ce admin admin] The token cache is being invalidate because project 41807e498b3947aca80280634840ef6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:23:54.789349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6ad0bd3f-2c84-4b37-b780-028396f175ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:54.790040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6ad0bd3f-2c84-4b37-b780-028396f175ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:54.791832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1037/2063] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:23:54 2022] DELETE /identity/v3/projects/41807e498b3947aca80280634840ef6b => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:54.795908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0bd3f-2c84-4b37-b780-028396f175ce admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:54.801111 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6ad0bd3f-2c84-4b37-b780-028396f175ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:54.828522 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1027/2064] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Oct 6 12:23:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:54.829496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865915aa-dcf3-4332-b49a-fb9e950f41e5 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:54.831593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-865915aa-dcf3-4332-b49a-fb9e950f41e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:54.884637 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:54.889438 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0bd3f-2c84-4b37-b780-028396f175ce admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:54.894322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22a4f9-3e31-4644-a306-5df79ecb4f24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:54.894746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22a4f9-3e31-4644-a306-5df79ecb4f24 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:54.895097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22a4f9-3e31-4644-a306-5df79ecb4f24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:54.895439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22a4f9-3e31-4644-a306-5df79ecb4f24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:54.898327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ac22a4f9-3e31-4644-a306-5df79ecb4f24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:54.917057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865915aa-dcf3-4332-b49a-fb9e950f41e5 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:54.921386 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180ac6e-bd46-4e5b-ab1c-5cbeec5e180c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:54.922000 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180ac6e-bd46-4e5b-ab1c-5cbeec5e180c service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:54.922284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180ac6e-bd46-4e5b-ab1c-5cbeec5e180c service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:54.922584 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180ac6e-bd46-4e5b-ab1c-5cbeec5e180c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:54.924388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8180ac6e-bd46-4e5b-ab1c-5cbeec5e180c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:54.976985 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22a4f9-3e31-4644-a306-5df79ecb4f24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:54.978684 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22a4f9-3e31-4644-a306-5df79ecb4f24 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:54.989072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8180ac6e-bd46-4e5b-ab1c-5cbeec5e180c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:54.989584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1038/2065] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:54.990620 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8180ac6e-bd46-4e5b-ab1c-5cbeec5e180c service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:54.990890 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7c7f3-3e89-4b80-8167-30b63acf88b8 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:54.991184 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7c7f3-3e89-4b80-8167-30b63acf88b8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:54.991458 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7c7f3-3e89-4b80-8167-30b63acf88b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.001786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1028/2066] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:55.003136 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90963ac-67ba-46df-84af-dd881dfdf04b None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.003431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90963ac-67ba-46df-84af-dd881dfdf04b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.003740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90963ac-67ba-46df-84af-dd881dfdf04b None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.004970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1029/2067] 10.222.0.33 () {54 vars in 961 bytes} [Thu Oct 6 12:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:23:55.011392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247bcbd-1d4a-475c-ac0e-5f63b166c69a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.011782 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247bcbd-1d4a-475c-ac0e-5f63b166c69a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.012134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247bcbd-1d4a-475c-ac0e-5f63b166c69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.016770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ffc7c7f3-3e89-4b80-8167-30b63acf88b8 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:55.018268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ffc7c7f3-3e89-4b80-8167-30b63acf88b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:55.027191 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1039/2068] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:55.036683 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9d13a-1a42-483d-8334-1ffb34497543 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.037250 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9d13a-1a42-483d-8334-1ffb34497543 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.037624 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9d13a-1a42-483d-8334-1ffb34497543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.038216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a247bcbd-1d4a-475c-ac0e-5f63b166c69a None None] MFA Rules not processed for user `39d67f5edf4b441ebaa2bb42e74b2cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:55.040498 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a247bcbd-1d4a-475c-ac0e-5f63b166c69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:55.050131 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1030/2069] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Oct 6 12:23:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:23:55.059261 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247bcbd-1d4a-475c-ac0e-5f63b166c69a None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:55.060818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-35f9d13a-1a42-483d-8334-1ffb34497543 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:55.062181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-35f9d13a-1a42-483d-8334-1ffb34497543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:55.070869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247bcbd-1d4a-475c-ac0e-5f63b166c69a None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:55.071295 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1040/2070] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:55.075152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9d13a-1a42-483d-8334-1ffb34497543 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:55.084023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816ccb3-a988-49f9-ba7d-114c1def58d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:55.084951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816ccb3-a988-49f9-ba7d-114c1def58d8 service placement] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.085944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816ccb3-a988-49f9-ba7d-114c1def58d8 service placement] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.086956 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816ccb3-a988-49f9-ba7d-114c1def58d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.089486 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9d13a-1a42-483d-8334-1ffb34497543 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:55.096748 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1cc0e-b049-4b57-b0df-3297b0362c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:55.097834 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1cc0e-b049-4b57-b0df-3297b0362c48 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.098310 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1cc0e-b049-4b57-b0df-3297b0362c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.098979 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1cc0e-b049-4b57-b0df-3297b0362c48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.099839 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1cc0e-b049-4b57-b0df-3297b0362c48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:23:55.100280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816ccb3-a988-49f9-ba7d-114c1def58d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:55.100713 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1cc0e-b049-4b57-b0df-3297b0362c48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:55.102303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816ccb3-a988-49f9-ba7d-114c1def58d8 service placement] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:55.103074 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1cc0e-b049-4b57-b0df-3297b0362c48 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:55.121545 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1041/2071] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:23:55.124286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1031/2072] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:55.128001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1cc0e-b049-4b57-b0df-3297b0362c48 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:55.141640 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1cc0e-b049-4b57-b0df-3297b0362c48 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:55.148608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea47954-6151-49e8-890b-04b69f22f0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:55.149217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea47954-6151-49e8-890b-04b69f22f0e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.149494 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea47954-6151-49e8-890b-04b69f22f0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.149814 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea47954-6151-49e8-890b-04b69f22f0e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.150450 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea47954-6151-49e8-890b-04b69f22f0e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:55.152561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea47954-6151-49e8-890b-04b69f22f0e0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:55.184981 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1042/2073] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:23:55 2022] POST /identity/v3/projects => generated 363 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:55.189140 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816ccb3-a988-49f9-ba7d-114c1def58d8 service placement] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:55.200441 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816ccb3-a988-49f9-ba7d-114c1def58d8 service placement] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:55.239563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684464d2-5cb9-407b-a1e1-2bcd99092980 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.239563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684464d2-5cb9-407b-a1e1-2bcd99092980 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.242355 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684464d2-5cb9-407b-a1e1-2bcd99092980 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.243799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1043/2074] 10.222.0.33 () {54 vars in 921 bytes} [Thu Oct 6 12:23:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:23:55.253021 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e67d9-0926-4d90-9e27-15d5142af0e8 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.253286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e67d9-0926-4d90-9e27-15d5142af0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.254006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e67d9-0926-4d90-9e27-15d5142af0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.301319 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-972e67d9-0926-4d90-9e27-15d5142af0e8 None None] MFA Rules not processed for user `39d67f5edf4b441ebaa2bb42e74b2cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:55.302619 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-972e67d9-0926-4d90-9e27-15d5142af0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:55.305700 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b72ff1-eb36-458b-9b84-f9e88e0c78b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:55.306327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b72ff1-eb36-458b-9b84-f9e88e0c78b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.306984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b72ff1-eb36-458b-9b84-f9e88e0c78b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.306984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b72ff1-eb36-458b-9b84-f9e88e0c78b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.307738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b72ff1-eb36-458b-9b84-f9e88e0c78b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:55.310515 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b72ff1-eb36-458b-9b84-f9e88e0c78b4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:55.337623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1032/2075] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:23:55 2022] POST /identity/v3/users => generated 282 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:55.339476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:55.340358 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b72ff1-eb36-458b-9b84-f9e88e0c78b4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:55.342461 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-59b72ff1-eb36-458b-9b84-f9e88e0c78b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:55.375668 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b72ff1-eb36-458b-9b84-f9e88e0c78b4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:55.380459 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48207fc6-c561-4106-bcb2-ceee27b9f011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:55.381099 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48207fc6-c561-4106-bcb2-ceee27b9f011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.381381 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48207fc6-c561-4106-bcb2-ceee27b9f011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.381650 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48207fc6-c561-4106-bcb2-ceee27b9f011 admin admin] PATH_INFO: `/v3/users/8ef1871bd4154fe18b167d1b69f2decd` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.382871 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48207fc6-c561-4106-bcb2-ceee27b9f011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef1871bd4154fe18b167d1b69f2decd)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:55.383406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1044/2076] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Oct 6 12:23:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:23:55.384433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e67d9-0926-4d90-9e27-15d5142af0e8 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:55.384785 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48207fc6-c561-4106-bcb2-ceee27b9f011 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:55.406130 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e67d9-0926-4d90-9e27-15d5142af0e8 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:55.416521 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab518a-6e4e-46d0-a3d2-49487d79d0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:55.417506 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab518a-6e4e-46d0-a3d2-49487d79d0ad admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.417792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab518a-6e4e-46d0-a3d2-49487d79d0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.418071 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab518a-6e4e-46d0-a3d2-49487d79d0ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.418733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab518a-6e4e-46d0-a3d2-49487d79d0ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:55.420438 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab518a-6e4e-46d0-a3d2-49487d79d0ad admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:55.437706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1045/2077] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:55.440816 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab518a-6e4e-46d0-a3d2-49487d79d0ad admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:55.467957 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab518a-6e4e-46d0-a3d2-49487d79d0ad admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:55.474200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf81d5-5208-472d-b3f4-dc4f0caa9bfa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:55.474822 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf81d5-5208-472d-b3f4-dc4f0caa9bfa service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.474971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:55.475129 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf81d5-5208-472d-b3f4-dc4f0caa9bfa service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.475453 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf81d5-5208-472d-b3f4-dc4f0caa9bfa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.480024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-48207fc6-c561-4106-bcb2-ceee27b9f011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef1871bd4154fe18b167d1b69f2decd'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:55.490900 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf81d5-5208-472d-b3f4-dc4f0caa9bfa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:55.493458 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf81d5-5208-472d-b3f4-dc4f0caa9bfa service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:55.506309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1046/2078] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:55.513240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-48207fc6-c561-4106-bcb2-ceee27b9f011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef1871bd4154fe18b167d1b69f2decd'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:55.513865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-48207fc6-c561-4106-bcb2-ceee27b9f011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef1871bd4154fe18b167d1b69f2decd'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:55.514497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-48207fc6-c561-4106-bcb2-ceee27b9f011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef1871bd4154fe18b167d1b69f2decd'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:55.517899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf81d5-5208-472d-b3f4-dc4f0caa9bfa service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:55.533756 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-48207fc6-c561-4106-bcb2-ceee27b9f011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef1871bd4154fe18b167d1b69f2decd'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:55.537892 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf81d5-5208-472d-b3f4-dc4f0caa9bfa service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:55.542886 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287eadf-2e59-4461-90e9-d85afadcc39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:55.543554 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287eadf-2e59-4461-90e9-d85afadcc39a admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.543842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287eadf-2e59-4461-90e9-d85afadcc39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.544117 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287eadf-2e59-4461-90e9-d85afadcc39a admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.544759 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b287eadf-2e59-4461-90e9-d85afadcc39a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:55.546580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b287eadf-2e59-4461-90e9-d85afadcc39a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:55.559642 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1047/2079] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:55.559642 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287eadf-2e59-4461-90e9-d85afadcc39a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:55.562346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b287eadf-2e59-4461-90e9-d85afadcc39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:55.575240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1033/2080] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:23:55 2022] DELETE /identity/v3/users/8ef1871bd4154fe18b167d1b69f2decd => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:55.576285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48207fc6-c561-4106-bcb2-ceee27b9f011 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:55.577852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-48207fc6-c561-4106-bcb2-ceee27b9f011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:55.579666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287eadf-2e59-4461-90e9-d85afadcc39a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:55.674435 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48207fc6-c561-4106-bcb2-ceee27b9f011 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:55.680935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905acb6e-fb07-4643-8f0b-e841fb54820c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:55.681224 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905acb6e-fb07-4643-8f0b-e841fb54820c admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.681508 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905acb6e-fb07-4643-8f0b-e841fb54820c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.682246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905acb6e-fb07-4643-8f0b-e841fb54820c admin admin] PATH_INFO: `/v3/projects/c1d6cd42c9874f7691bff03bb4375b5e/users/e911cf941fbb47dea9bb0946a446177c/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.701523 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638efc40-b100-47f2-9b0b-52623934f099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:55.702216 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638efc40-b100-47f2-9b0b-52623934f099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.702505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638efc40-b100-47f2-9b0b-52623934f099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.702781 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638efc40-b100-47f2-9b0b-52623934f099 admin admin] PATH_INFO: `/v3/projects/0610c3c8314f4212b848fd79761cf145` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.704652 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638efc40-b100-47f2-9b0b-52623934f099 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0610c3c8314f4212b848fd79761cf145)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:55.708081 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638efc40-b100-47f2-9b0b-52623934f099 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:55.708564 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905acb6e-fb07-4643-8f0b-e841fb54820c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1d6cd42c9874f7691bff03bb4375b5e, user_id=e911cf941fbb47dea9bb0946a446177c, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:55.712286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905acb6e-fb07-4643-8f0b-e841fb54820c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:55.733978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1034/2081] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:55 2022] PUT /identity/v3/projects/c1d6cd42c9874f7691bff03bb4375b5e/users/e911cf941fbb47dea9bb0946a446177c/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:55.735597 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905acb6e-fb07-4643-8f0b-e841fb54820c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:55.737475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-905acb6e-fb07-4643-8f0b-e841fb54820c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:55.804104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-638efc40-b100-47f2-9b0b-52623934f099 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0610c3c8314f4212b848fd79761cf145', 'request_id': 'req-638efc40-b100-47f2-9b0b-52623934f099'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:55.836604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-638efc40-b100-47f2-9b0b-52623934f099 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0610c3c8314f4212b848fd79761cf145', 'request_id': 'req-638efc40-b100-47f2-9b0b-52623934f099'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:55.844729 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-638efc40-b100-47f2-9b0b-52623934f099 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0610c3c8314f4212b848fd79761cf145', 'request_id': 'req-638efc40-b100-47f2-9b0b-52623934f099'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:55.867561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-638efc40-b100-47f2-9b0b-52623934f099 admin admin] The token cache is being invalidate because project 0610c3c8314f4212b848fd79761cf145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:23:55.868817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-638efc40-b100-47f2-9b0b-52623934f099 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:55.869617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-638efc40-b100-47f2-9b0b-52623934f099 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:55.884433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1048/2082] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:23:55 2022] DELETE /identity/v3/projects/0610c3c8314f4212b848fd79761cf145 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:55.885970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d06de-9ad4-4832-9d44-5cf90dc305cc None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.886253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d06de-9ad4-4832-9d44-5cf90dc305cc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.886526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d06de-9ad4-4832-9d44-5cf90dc305cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.898326 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905acb6e-fb07-4643-8f0b-e841fb54820c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:55.960831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b13d06de-9ad4-4832-9d44-5cf90dc305cc None None] MFA Rules not processed for user `e911cf941fbb47dea9bb0946a446177c`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:55.962056 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b13d06de-9ad4-4832-9d44-5cf90dc305cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:55.967565 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:55.988521 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d44cb-4d56-4260-98d0-7caefa044061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:55.989465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d44cb-4d56-4260-98d0-7caefa044061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:55.989749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d44cb-4d56-4260-98d0-7caefa044061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:55.990024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d44cb-4d56-4260-98d0-7caefa044061 admin admin] PATH_INFO: `/v3/projects/d785e123e16648908d42c04a0130dc0e` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:55.992669 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:55.993354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d44cb-4d56-4260-98d0-7caefa044061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d785e123e16648908d42c04a0130dc0e)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:55.997866 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d44cb-4d56-4260-98d0-7caefa044061 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:56.079128 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1049/2083] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:56.079128 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d06de-9ad4-4832-9d44-5cf90dc305cc None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:56.082697 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b13d06de-9ad4-4832-9d44-5cf90dc305cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:56.094080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-513d44cb-4d56-4260-98d0-7caefa044061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd785e123e16648908d42c04a0130dc0e', 'request_id': 'req-513d44cb-4d56-4260-98d0-7caefa044061'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:56.100768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d06de-9ad4-4832-9d44-5cf90dc305cc None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:56.116070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-513d44cb-4d56-4260-98d0-7caefa044061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd785e123e16648908d42c04a0130dc0e', 'request_id': 'req-513d44cb-4d56-4260-98d0-7caefa044061'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:56.128875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-513d44cb-4d56-4260-98d0-7caefa044061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd785e123e16648908d42c04a0130dc0e', 'request_id': 'req-513d44cb-4d56-4260-98d0-7caefa044061'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:56.150954 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-513d44cb-4d56-4260-98d0-7caefa044061 admin admin] The token cache is being invalidate because project d785e123e16648908d42c04a0130dc0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:23:56.150954 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-513d44cb-4d56-4260-98d0-7caefa044061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:56.151920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-513d44cb-4d56-4260-98d0-7caefa044061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:56.154710 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1035/2084] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:23:55 2022] DELETE /identity/v3/projects/d785e123e16648908d42c04a0130dc0e => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:56.159403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea8572-9af3-4313-8bc4-153db7b3a561 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:56.159733 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea8572-9af3-4313-8bc4-153db7b3a561 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:56.160078 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea8572-9af3-4313-8bc4-153db7b3a561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:56.181242 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-47ea8572-9af3-4313-8bc4-153db7b3a561 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:56.182394 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-47ea8572-9af3-4313-8bc4-153db7b3a561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:56.189781 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bca2e2-669d-4656-809f-4ee9bc954672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:56.190481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bca2e2-669d-4656-809f-4ee9bc954672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:56.190851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bca2e2-669d-4656-809f-4ee9bc954672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:56.191200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bca2e2-669d-4656-809f-4ee9bc954672 admin admin] PATH_INFO: `/v3/projects/2a97e649b405464a9fc7ec379401ca2c` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:56.192413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bca2e2-669d-4656-809f-4ee9bc954672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a97e649b405464a9fc7ec379401ca2c)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:56.194424 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bca2e2-669d-4656-809f-4ee9bc954672 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:56.256457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1036/2085] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:56.258111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea8572-9af3-4313-8bc4-153db7b3a561 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:56.259620 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-47ea8572-9af3-4313-8bc4-153db7b3a561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:56.263735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e9bca2e2-669d-4656-809f-4ee9bc954672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a97e649b405464a9fc7ec379401ca2c', 'request_id': 'req-e9bca2e2-669d-4656-809f-4ee9bc954672'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:56.271491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea8572-9af3-4313-8bc4-153db7b3a561 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:56.273762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e9bca2e2-669d-4656-809f-4ee9bc954672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a97e649b405464a9fc7ec379401ca2c', 'request_id': 'req-e9bca2e2-669d-4656-809f-4ee9bc954672'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:56.281453 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e9bca2e2-669d-4656-809f-4ee9bc954672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a97e649b405464a9fc7ec379401ca2c', 'request_id': 'req-e9bca2e2-669d-4656-809f-4ee9bc954672'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:56.293172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e9bca2e2-669d-4656-809f-4ee9bc954672 admin admin] The token cache is being invalidate because project 2a97e649b405464a9fc7ec379401ca2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:23:56.293464 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e9bca2e2-669d-4656-809f-4ee9bc954672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:56.294157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e9bca2e2-669d-4656-809f-4ee9bc954672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:56.295710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1050/2086] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:23:56 2022] DELETE /identity/v3/projects/2a97e649b405464a9fc7ec379401ca2c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:56.297007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f81e8-212e-4804-b580-fe0c8627c763 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:56.297287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f81e8-212e-4804-b580-fe0c8627c763 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:56.297559 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f81e8-212e-4804-b580-fe0c8627c763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:56.318102 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-4d1f81e8-212e-4804-b580-fe0c8627c763 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:56.319327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4d1f81e8-212e-4804-b580-fe0c8627c763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:56.348961 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d82e1-9a3d-4f56-b67f-ebf2b35d461c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:56.349608 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d82e1-9a3d-4f56-b67f-ebf2b35d461c admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:56.350009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d82e1-9a3d-4f56-b67f-ebf2b35d461c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:56.350316 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d82e1-9a3d-4f56-b67f-ebf2b35d461c admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:56.350947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d82e1-9a3d-4f56-b67f-ebf2b35d461c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:56.354220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d82e1-9a3d-4f56-b67f-ebf2b35d461c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:56.370955 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:56.384040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1037/2087] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:23:56 2022] POST /identity/v3/projects => generated 363 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:56.385304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94771118-500f-4cf9-a229-5a63e4f5112f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:56.385595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94771118-500f-4cf9-a229-5a63e4f5112f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:56.385870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94771118-500f-4cf9-a229-5a63e4f5112f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:56.394818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1051/2088] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:56.396345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:56.396608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f81e8-212e-4804-b580-fe0c8627c763 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:56.404889 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4d1f81e8-212e-4804-b580-fe0c8627c763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:56.409640 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-94771118-500f-4cf9-a229-5a63e4f5112f None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:56.411029 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-94771118-500f-4cf9-a229-5a63e4f5112f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:56.420220 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f81e8-212e-4804-b580-fe0c8627c763 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:56.482505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813485c9-0a79-4597-bcf8-ec7a6f53befb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:56.483348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813485c9-0a79-4597-bcf8-ec7a6f53befb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:56.483348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813485c9-0a79-4597-bcf8-ec7a6f53befb admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:56.483633 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813485c9-0a79-4597-bcf8-ec7a6f53befb admin admin] PATH_INFO: `/v3/users/e39a28e74bfe43d1896e5d1f3af8d205` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:56.483738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1038/2089] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:56.484915 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813485c9-0a79-4597-bcf8-ec7a6f53befb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e39a28e74bfe43d1896e5d1f3af8d205)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:56.487185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813485c9-0a79-4597-bcf8-ec7a6f53befb admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:56.487840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94771118-500f-4cf9-a229-5a63e4f5112f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:56.490431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-94771118-500f-4cf9-a229-5a63e4f5112f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:56.503279 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94771118-500f-4cf9-a229-5a63e4f5112f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:56.509153 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a0440-9527-4902-be69-75c77068fdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:56.509812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a0440-9527-4902-be69-75c77068fdee admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:56.510088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a0440-9527-4902-be69-75c77068fdee admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:56.510352 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a0440-9527-4902-be69-75c77068fdee admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:56.510977 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a0440-9527-4902-be69-75c77068fdee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:56.512922 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a0440-9527-4902-be69-75c77068fdee admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:56.542583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1039/2090] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:23:56 2022] POST /identity/v3/users => generated 288 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:56.542583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d02e2-707f-4397-87cd-127f247038ee None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:56.542583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d02e2-707f-4397-87cd-127f247038ee None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:56.542583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d02e2-707f-4397-87cd-127f247038ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:56.545983 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-813485c9-0a79-4597-bcf8-ec7a6f53befb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e39a28e74bfe43d1896e5d1f3af8d205'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:56.554256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-813485c9-0a79-4597-bcf8-ec7a6f53befb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e39a28e74bfe43d1896e5d1f3af8d205'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:56.555496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-813485c9-0a79-4597-bcf8-ec7a6f53befb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e39a28e74bfe43d1896e5d1f3af8d205'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:56.556589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-813485c9-0a79-4597-bcf8-ec7a6f53befb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e39a28e74bfe43d1896e5d1f3af8d205'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:56.561930 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-783d02e2-707f-4397-87cd-127f247038ee None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:56.563637 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-783d02e2-707f-4397-87cd-127f247038ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:56.566067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-813485c9-0a79-4597-bcf8-ec7a6f53befb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e39a28e74bfe43d1896e5d1f3af8d205'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:56.576226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1040/2091] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:56.576226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d02e2-707f-4397-87cd-127f247038ee None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:56.576226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-783d02e2-707f-4397-87cd-127f247038ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:56.587948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d02e2-707f-4397-87cd-127f247038ee None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:56.590059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1052/2092] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:23:56 2022] DELETE /identity/v3/users/e39a28e74bfe43d1896e5d1f3af8d205 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:56.590672 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813485c9-0a79-4597-bcf8-ec7a6f53befb admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:56.592010 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-813485c9-0a79-4597-bcf8-ec7a6f53befb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:56.674223 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b93e8-78cc-41b8-ba17-bc1d4d05eb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:56.674843 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b93e8-78cc-41b8-ba17-bc1d4d05eb98 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:56.675122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b93e8-78cc-41b8-ba17-bc1d4d05eb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:56.675518 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b93e8-78cc-41b8-ba17-bc1d4d05eb98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:56.676186 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b93e8-78cc-41b8-ba17-bc1d4d05eb98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:23:56.676487 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b93e8-78cc-41b8-ba17-bc1d4d05eb98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:56.678663 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813485c9-0a79-4597-bcf8-ec7a6f53befb admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:56.678796 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b93e8-78cc-41b8-ba17-bc1d4d05eb98 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:56.684754 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22c6cf-29b5-43e7-a547-69c70893d0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:56.685648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22c6cf-29b5-43e7-a547-69c70893d0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:56.686128 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22c6cf-29b5-43e7-a547-69c70893d0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:56.686594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22c6cf-29b5-43e7-a547-69c70893d0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:56.687436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22c6cf-29b5-43e7-a547-69c70893d0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:56.689428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22c6cf-29b5-43e7-a547-69c70893d0e9 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:56.699111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1041/2093] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:23:56.700070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b93e8-78cc-41b8-ba17-bc1d4d05eb98 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:56.708756 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1053/2094] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:56.710241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22c6cf-29b5-43e7-a547-69c70893d0e9 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:56.711547 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b93e8-78cc-41b8-ba17-bc1d4d05eb98 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:56.717069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac424f-a06a-4fec-99b1-449836c884d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:56.717675 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac424f-a06a-4fec-99b1-449836c884d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:56.718452 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac424f-a06a-4fec-99b1-449836c884d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:56.718728 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac424f-a06a-4fec-99b1-449836c884d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:56.719369 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:56.719494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac424f-a06a-4fec-99b1-449836c884d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:23:56.719831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac424f-a06a-4fec-99b1-449836c884d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:56.722727 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac424f-a06a-4fec-99b1-449836c884d5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:56.725080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22c6cf-29b5-43e7-a547-69c70893d0e9 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:56.731249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9b0bc-46ab-4ccb-8856-dbbec08ab3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:56.732295 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9b0bc-46ab-4ccb-8856-dbbec08ab3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:56.733021 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9b0bc-46ab-4ccb-8856-dbbec08ab3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:56.733549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9b0bc-46ab-4ccb-8856-dbbec08ab3a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:56.734403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b9b0bc-46ab-4ccb-8856-dbbec08ab3a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:56.738150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b9b0bc-46ab-4ccb-8856-dbbec08ab3a5 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:56.745707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1042/2095] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:23:56.745707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac424f-a06a-4fec-99b1-449836c884d5 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:56.760852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac424f-a06a-4fec-99b1-449836c884d5 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:56.767559 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a894db0-274c-4c13-8241-76ae4b838820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:56.768554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a894db0-274c-4c13-8241-76ae4b838820 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:56.769334 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a894db0-274c-4c13-8241-76ae4b838820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:56.769334 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a894db0-274c-4c13-8241-76ae4b838820 admin admin] PATH_INFO: `/v3/projects/31128bb6082342fc956fb5ccbbe02b03/users/134bede3d73845ccbb676fe0bcdd00b7/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:56.796700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1054/2096] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:23:56 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:56.796700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9b0bc-46ab-4ccb-8856-dbbec08ab3a5 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:56.798651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a894db0-274c-4c13-8241-76ae4b838820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31128bb6082342fc956fb5ccbbe02b03, user_id=134bede3d73845ccbb676fe0bcdd00b7, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:56.801745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a894db0-274c-4c13-8241-76ae4b838820 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:56.835158 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1043/2097] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:56 2022] PUT /identity/v3/projects/31128bb6082342fc956fb5ccbbe02b03/users/134bede3d73845ccbb676fe0bcdd00b7/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:56.836999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a894db0-274c-4c13-8241-76ae4b838820 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:56.892125 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9b0bc-46ab-4ccb-8856-dbbec08ab3a5 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:56.894911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:56.926439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a894db0-274c-4c13-8241-76ae4b838820 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:56.933090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217513f-9edd-48bf-b2b2-f2ee98963512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:56.933731 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217513f-9edd-48bf-b2b2-f2ee98963512 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:56.933994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217513f-9edd-48bf-b2b2-f2ee98963512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:56.934261 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217513f-9edd-48bf-b2b2-f2ee98963512 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:56.934902 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217513f-9edd-48bf-b2b2-f2ee98963512 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:56.937128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217513f-9edd-48bf-b2b2-f2ee98963512 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:56.971719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1044/2098] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:23:56 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:56.971719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217513f-9edd-48bf-b2b2-f2ee98963512 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:56.973986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4217513f-9edd-48bf-b2b2-f2ee98963512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:56.986557 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7193f0-c4eb-4d72-87cc-75146a6473cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:56.987103 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7193f0-c4eb-4d72-87cc-75146a6473cd admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:56.987384 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7193f0-c4eb-4d72-87cc-75146a6473cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:56.987529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217513f-9edd-48bf-b2b2-f2ee98963512 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:56.987742 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7193f0-c4eb-4d72-87cc-75146a6473cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:56.988498 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7193f0-c4eb-4d72-87cc-75146a6473cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:56.990369 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7193f0-c4eb-4d72-87cc-75146a6473cd admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:56.993045 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd453213-6bfc-4364-a889-5e3d3aa4dba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:56.994285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd453213-6bfc-4364-a889-5e3d3aa4dba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:56.994570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd453213-6bfc-4364-a889-5e3d3aa4dba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:56.994853 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd453213-6bfc-4364-a889-5e3d3aa4dba2 admin admin] PATH_INFO: `/v3/projects/71b5f1c8eb874fc2924e02f0950d0fed` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:56.996883 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd453213-6bfc-4364-a889-5e3d3aa4dba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b5f1c8eb874fc2924e02f0950d0fed)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:56.998892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd453213-6bfc-4364-a889-5e3d3aa4dba2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:57.019628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1055/2099] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:23:56 2022] POST /identity/v3/projects => generated 383 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:57.020550 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7193f0-c4eb-4d72-87cc-75146a6473cd admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:57.064184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cd453213-6bfc-4364-a889-5e3d3aa4dba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b5f1c8eb874fc2924e02f0950d0fed', 'request_id': 'req-cd453213-6bfc-4364-a889-5e3d3aa4dba2'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:57.073628 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cd453213-6bfc-4364-a889-5e3d3aa4dba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b5f1c8eb874fc2924e02f0950d0fed', 'request_id': 'req-cd453213-6bfc-4364-a889-5e3d3aa4dba2'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:57.082471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cd453213-6bfc-4364-a889-5e3d3aa4dba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b5f1c8eb874fc2924e02f0950d0fed', 'request_id': 'req-cd453213-6bfc-4364-a889-5e3d3aa4dba2'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:57.089498 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7193f0-c4eb-4d72-87cc-75146a6473cd admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:57.101848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cd453213-6bfc-4364-a889-5e3d3aa4dba2 admin admin] The token cache is being invalidate because project 71b5f1c8eb874fc2924e02f0950d0fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:23:57.102114 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cd453213-6bfc-4364-a889-5e3d3aa4dba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:57.102808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cd453213-6bfc-4364-a889-5e3d3aa4dba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:23:57.109345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1045/2100] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:23:56 2022] DELETE /identity/v3/projects/71b5f1c8eb874fc2924e02f0950d0fed => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:57.109345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd453213-6bfc-4364-a889-5e3d3aa4dba2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:57.109345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-cd453213-6bfc-4364-a889-5e3d3aa4dba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:57.121549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:57.192055 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad895036-4619-4ff2-bf41-3273ae9f0216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:57.193041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad895036-4619-4ff2-bf41-3273ae9f0216 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.193155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad895036-4619-4ff2-bf41-3273ae9f0216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.193595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad895036-4619-4ff2-bf41-3273ae9f0216 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.194261 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad895036-4619-4ff2-bf41-3273ae9f0216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:57.197113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad895036-4619-4ff2-bf41-3273ae9f0216 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:57.208110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd453213-6bfc-4364-a889-5e3d3aa4dba2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:57.213570 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1056/2101] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:57.214816 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad895036-4619-4ff2-bf41-3273ae9f0216 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:57.215148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c2d14-0536-4938-9027-6344cad38d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:57.215885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c2d14-0536-4938-9027-6344cad38d12 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.216168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c2d14-0536-4938-9027-6344cad38d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.216442 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c2d14-0536-4938-9027-6344cad38d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.217093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c2d14-0536-4938-9027-6344cad38d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:57.217995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ad895036-4619-4ff2-bf41-3273ae9f0216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:57.219022 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c2d14-0536-4938-9027-6344cad38d12 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:57.229867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1046/2102] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:57.230995 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1b4c7-3930-4e21-b31f-57eac81105db None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.231284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1b4c7-3930-4e21-b31f-57eac81105db None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.231582 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1b4c7-3930-4e21-b31f-57eac81105db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.254483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-21d1b4c7-3930-4e21-b31f-57eac81105db None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:57.255885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-21d1b4c7-3930-4e21-b31f-57eac81105db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:57.257305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad895036-4619-4ff2-bf41-3273ae9f0216 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:57.262282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e5aab-1af7-4dd2-a296-fe598633f372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:57.262893 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e5aab-1af7-4dd2-a296-fe598633f372 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.263696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e5aab-1af7-4dd2-a296-fe598633f372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.263696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e5aab-1af7-4dd2-a296-fe598633f372 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.264072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1047/2103] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:57.270550 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e5aab-1af7-4dd2-a296-fe598633f372 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:57.270669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1b4c7-3930-4e21-b31f-57eac81105db None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:57.270669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-21d1b4c7-3930-4e21-b31f-57eac81105db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:57.270837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e5aab-1af7-4dd2-a296-fe598633f372 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:57.278903 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1b4c7-3930-4e21-b31f-57eac81105db None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:57.284572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d45fc-66ac-4c1a-bcc4-228221e0077f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:57.285667 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d45fc-66ac-4c1a-bcc4-228221e0077f admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.285799 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d45fc-66ac-4c1a-bcc4-228221e0077f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.286206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d45fc-66ac-4c1a-bcc4-228221e0077f admin admin] PATH_INFO: `/v3/projects/31128bb6082342fc956fb5ccbbe02b03/users/134bede3d73845ccbb676fe0bcdd00b7/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.288261 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d45fc-66ac-4c1a-bcc4-228221e0077f admin admin] RBAC: Authorizing `identity:create_grant(project_id=31128bb6082342fc956fb5ccbbe02b03, user_id=134bede3d73845ccbb676fe0bcdd00b7, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:57.292347 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d45fc-66ac-4c1a-bcc4-228221e0077f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:57.298915 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1057/2104] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:23:57 2022] POST /identity/v3/users => generated 292 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:57.300515 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e5aab-1af7-4dd2-a296-fe598633f372 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:57.313315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1048/2105] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:57 2022] PUT /identity/v3/projects/31128bb6082342fc956fb5ccbbe02b03/users/134bede3d73845ccbb676fe0bcdd00b7/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:57.314765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e5aab-1af7-4dd2-a296-fe598633f372 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:57.314894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec1f16-f892-442b-8c11-d02a4c333fb8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.315072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec1f16-f892-442b-8c11-d02a4c333fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.315334 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec1f16-f892-442b-8c11-d02a4c333fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.336322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-adec1f16-f892-442b-8c11-d02a4c333fb8 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:57.337565 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-adec1f16-f892-442b-8c11-d02a4c333fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:57.360766 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:57.391468 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b166558-27b0-4161-84a5-5d04937be51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:57.392254 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b166558-27b0-4161-84a5-5d04937be51f admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.392745 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b166558-27b0-4161-84a5-5d04937be51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.393428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b166558-27b0-4161-84a5-5d04937be51f admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.394174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b166558-27b0-4161-84a5-5d04937be51f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:57.395894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b166558-27b0-4161-84a5-5d04937be51f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:57.404851 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1049/2106] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:57.408799 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec1f16-f892-442b-8c11-d02a4c333fb8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:57.408903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1058/2107] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:57.410632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb59eff-7a21-4c69-83b1-f38cbaa4a6ae None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.411002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb59eff-7a21-4c69-83b1-f38cbaa4a6ae None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.411477 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb59eff-7a21-4c69-83b1-f38cbaa4a6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.417828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec1f16-f892-442b-8c11-d02a4c333fb8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:57.423859 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ee147-7b49-46f9-9e9f-f86796d58283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:57.424560 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ee147-7b49-46f9-9e9f-f86796d58283 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.424956 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ee147-7b49-46f9-9e9f-f86796d58283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.425243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ee147-7b49-46f9-9e9f-f86796d58283 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.426320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ee147-7b49-46f9-9e9f-f86796d58283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:57.427986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ee147-7b49-46f9-9e9f-f86796d58283 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:57.439538 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1050/2108] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:57.440311 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ee147-7b49-46f9-9e9f-f86796d58283 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:57.443764 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-dcb59eff-7a21-4c69-83b1-f38cbaa4a6ae None None] MFA Rules not processed for user `134bede3d73845ccbb676fe0bcdd00b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:57.445644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-dcb59eff-7a21-4c69-83b1-f38cbaa4a6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:57.450915 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ee147-7b49-46f9-9e9f-f86796d58283 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:57.456313 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87218a6-5dd1-4e0b-9604-f38e81924939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:57.456886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87218a6-5dd1-4e0b-9604-f38e81924939 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.457163 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87218a6-5dd1-4e0b-9604-f38e81924939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.457441 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87218a6-5dd1-4e0b-9604-f38e81924939 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.458085 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87218a6-5dd1-4e0b-9604-f38e81924939 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:23:57.458391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87218a6-5dd1-4e0b-9604-f38e81924939 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:57.460062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87218a6-5dd1-4e0b-9604-f38e81924939 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:57.473269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1051/2109] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:23:57.474095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87218a6-5dd1-4e0b-9604-f38e81924939 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:57.483846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87218a6-5dd1-4e0b-9604-f38e81924939 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:57.488537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd20df-71d1-4197-9696-549ea5682966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:57.489200 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd20df-71d1-4197-9696-549ea5682966 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.489482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd20df-71d1-4197-9696-549ea5682966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.489759 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd20df-71d1-4197-9696-549ea5682966 admin admin] PATH_INFO: `/v3/projects/9bc37605c3444676a9783bcac463c2d5/users/89f04190e02b446a94a7cf81ab1d5b16/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.511345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd20df-71d1-4197-9696-549ea5682966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc37605c3444676a9783bcac463c2d5, user_id=89f04190e02b446a94a7cf81ab1d5b16, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:57.514074 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd20df-71d1-4197-9696-549ea5682966 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:57.534033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1052/2110] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:57 2022] PUT /identity/v3/projects/9bc37605c3444676a9783bcac463c2d5/users/89f04190e02b446a94a7cf81ab1d5b16/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:57.534876 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd20df-71d1-4197-9696-549ea5682966 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:57.557369 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1059/2111] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:57.558479 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb59eff-7a21-4c69-83b1-f38cbaa4a6ae None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:57.572181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb59eff-7a21-4c69-83b1-f38cbaa4a6ae None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:57.597043 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:57.633058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd20df-71d1-4197-9696-549ea5682966 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:57.638139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c7f9c-2ced-4336-91ee-862745cfb796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:57.638880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c7f9c-2ced-4336-91ee-862745cfb796 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.639219 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c7f9c-2ced-4336-91ee-862745cfb796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.639528 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c7f9c-2ced-4336-91ee-862745cfb796 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.640872 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c7f9c-2ced-4336-91ee-862745cfb796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:57.640995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8335ed-05be-418e-ab9b-e1de496da67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:57.640995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8335ed-05be-418e-ab9b-e1de496da67a admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.641170 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8335ed-05be-418e-ab9b-e1de496da67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.641549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8335ed-05be-418e-ab9b-e1de496da67a admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.641926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c7f9c-2ced-4336-91ee-862745cfb796 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:57.642342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8335ed-05be-418e-ab9b-e1de496da67a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:57.644519 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8335ed-05be-418e-ab9b-e1de496da67a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:57.656214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1053/2112] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:57.656214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c440a4-9c92-4181-9f22-2d307c2a7b22 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.656214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c440a4-9c92-4181-9f22-2d307c2a7b22 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.656214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c440a4-9c92-4181-9f22-2d307c2a7b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.675744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1060/2113] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:23:57 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:57.680002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8335ed-05be-418e-ab9b-e1de496da67a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:57.681693 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-da8335ed-05be-418e-ab9b-e1de496da67a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:57.687576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d3c440a4-9c92-4181-9f22-2d307c2a7b22 None None] MFA Rules not processed for user `89f04190e02b446a94a7cf81ab1d5b16`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:57.689679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d3c440a4-9c92-4181-9f22-2d307c2a7b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:57.742631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8335ed-05be-418e-ab9b-e1de496da67a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:57.746294 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023b7cf-f86a-4d94-9714-032a8c749346 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:57.746967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023b7cf-f86a-4d94-9714-032a8c749346 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.747177 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023b7cf-f86a-4d94-9714-032a8c749346 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.747445 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023b7cf-f86a-4d94-9714-032a8c749346 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.756290 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1054/2114] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:57.757978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c440a4-9c92-4181-9f22-2d307c2a7b22 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:57.764387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:57.817934 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023b7cf-f86a-4d94-9714-032a8c749346 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:57.819498 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023b7cf-f86a-4d94-9714-032a8c749346 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:57.828596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c440a4-9c92-4181-9f22-2d307c2a7b22 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:57.830157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1061/2115] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:57.832837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023b7cf-f86a-4d94-9714-032a8c749346 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:57.834311 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d6506-1a86-4529-b4c7-8582d6e594eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:57.834982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d6506-1a86-4529-b4c7-8582d6e594eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.835251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d6506-1a86-4529-b4c7-8582d6e594eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.835587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d6506-1a86-4529-b4c7-8582d6e594eb admin admin] PATH_INFO: `/v3/projects/468a5076e9994c5bb5691cc1146b1c22/users/3dc1230116cb46d9a3a4110cabf6caf5/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.847931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023b7cf-f86a-4d94-9714-032a8c749346 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:57.856189 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2c2e8-b8e2-4a14-9df8-2bc85fd19b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:57.856815 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2c2e8-b8e2-4a14-9df8-2bc85fd19b2b admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.857110 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2c2e8-b8e2-4a14-9df8-2bc85fd19b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.857372 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2c2e8-b8e2-4a14-9df8-2bc85fd19b2b admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.858005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2c2e8-b8e2-4a14-9df8-2bc85fd19b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:57.859986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2c2e8-b8e2-4a14-9df8-2bc85fd19b2b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:57.860257 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d6506-1a86-4529-b4c7-8582d6e594eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=468a5076e9994c5bb5691cc1146b1c22, user_id=3dc1230116cb46d9a3a4110cabf6caf5, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:57.863004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d6506-1a86-4529-b4c7-8582d6e594eb admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:57.890082 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1055/2116] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:57 2022] PUT /identity/v3/projects/468a5076e9994c5bb5691cc1146b1c22/users/3dc1230116cb46d9a3a4110cabf6caf5/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:57.890082 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d6506-1a86-4529-b4c7-8582d6e594eb admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:57.893438 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1062/2117] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:23:57 2022] POST /identity/v3/users => generated 285 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:57.894711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54ef3a-dcfb-4039-8403-817fb419ce82 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.895001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54ef3a-dcfb-4039-8403-817fb419ce82 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.895396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54ef3a-dcfb-4039-8403-817fb419ce82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.929788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fc54ef3a-dcfb-4039-8403-817fb419ce82 None None] MFA Rules not processed for user `3dc1230116cb46d9a3a4110cabf6caf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:57.931007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fc54ef3a-dcfb-4039-8403-817fb419ce82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:57.966330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:57.966981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d6506-1a86-4529-b4c7-8582d6e594eb admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:57.974094 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d19db6-ae6f-4d9d-a570-99460714d548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:57.974947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d19db6-ae6f-4d9d-a570-99460714d548 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:57.975232 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d19db6-ae6f-4d9d-a570-99460714d548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:57.975612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d19db6-ae6f-4d9d-a570-99460714d548 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:57.976278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d19db6-ae6f-4d9d-a570-99460714d548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:57.978392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d19db6-ae6f-4d9d-a570-99460714d548 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:58.011569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1056/2118] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:23:57 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:58.012923 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d19db6-ae6f-4d9d-a570-99460714d548 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:58.020717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1063/2119] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:58.020717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54ef3a-dcfb-4039-8403-817fb419ce82 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:58.076421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54ef3a-dcfb-4039-8403-817fb419ce82 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:58.100336 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d19db6-ae6f-4d9d-a570-99460714d548 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:58.105568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04f3da-db4a-4ac0-aba5-d595917c8a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:58.106364 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04f3da-db4a-4ac0-aba5-d595917c8a1c admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.106710 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04f3da-db4a-4ac0-aba5-d595917c8a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.107050 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04f3da-db4a-4ac0-aba5-d595917c8a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.107793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04f3da-db4a-4ac0-aba5-d595917c8a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:58.109674 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04f3da-db4a-4ac0-aba5-d595917c8a1c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:58.123125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1057/2120] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:58.123125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04f3da-db4a-4ac0-aba5-d595917c8a1c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:58.134260 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04f3da-db4a-4ac0-aba5-d595917c8a1c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:58.137291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de98cc-afa5-45fa-9937-eb1b54e416f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:58.138346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de98cc-afa5-45fa-9937-eb1b54e416f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.138799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de98cc-afa5-45fa-9937-eb1b54e416f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.139040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1de156-6271-4033-bc82-178a87b52b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:58.139226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de98cc-afa5-45fa-9937-eb1b54e416f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.139673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1de156-6271-4033-bc82-178a87b52b59 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.139951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1de156-6271-4033-bc82-178a87b52b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.140220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1de156-6271-4033-bc82-178a87b52b59 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.140844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1de156-6271-4033-bc82-178a87b52b59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:58.141249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c7de98cc-afa5-45fa-9937-eb1b54e416f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:58.142738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1de156-6271-4033-bc82-178a87b52b59 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:58.152639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:58.167551 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1058/2121] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:23:58 2022] POST /identity/v3/users => generated 293 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:58.168754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1de156-6271-4033-bc82-178a87b52b59 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:58.179636 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1de156-6271-4033-bc82-178a87b52b59 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:58.184566 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62c9d4-3295-43c2-b286-7fa1e5992886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:58.185166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62c9d4-3295-43c2-b286-7fa1e5992886 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.185451 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62c9d4-3295-43c2-b286-7fa1e5992886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.185726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62c9d4-3295-43c2-b286-7fa1e5992886 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.186361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62c9d4-3295-43c2-b286-7fa1e5992886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:58.188367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62c9d4-3295-43c2-b286-7fa1e5992886 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:58.213238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de98cc-afa5-45fa-9937-eb1b54e416f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:58.214926 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de98cc-afa5-45fa-9937-eb1b54e416f4 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:58.223343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1059/2122] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:23:58 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:58.223343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62c9d4-3295-43c2-b286-7fa1e5992886 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:58.228843 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1064/2123] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:58.229873 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de98cc-afa5-45fa-9937-eb1b54e416f4 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:58.255386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de98cc-afa5-45fa-9937-eb1b54e416f4 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:58.313925 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62c9d4-3295-43c2-b286-7fa1e5992886 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:58.314943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:58.320772 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbd04a-a266-43a6-bb01-d14422bedbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:58.321454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbd04a-a266-43a6-bb01-d14422bedbfe admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.321738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbd04a-a266-43a6-bb01-d14422bedbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.322021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbd04a-a266-43a6-bb01-d14422bedbfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.322730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbd04a-a266-43a6-bb01-d14422bedbfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:58.324572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbd04a-a266-43a6-bb01-d14422bedbfe admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:58.338389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1060/2124] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:58.339367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbd04a-a266-43a6-bb01-d14422bedbfe admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:58.344583 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab27be-a0ff-42ee-9a6c-536e0c446a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:58.345514 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab27be-a0ff-42ee-9a6c-536e0c446a11 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.345888 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab27be-a0ff-42ee-9a6c-536e0c446a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.347083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab27be-a0ff-42ee-9a6c-536e0c446a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.348558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab27be-a0ff-42ee-9a6c-536e0c446a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:58.352922 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab27be-a0ff-42ee-9a6c-536e0c446a11 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:58.358354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbd04a-a266-43a6-bb01-d14422bedbfe admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:58.363459 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462af5b-bd38-4b2f-bcd8-087154b37b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:58.364108 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462af5b-bd38-4b2f-bcd8-087154b37b7d admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.364446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462af5b-bd38-4b2f-bcd8-087154b37b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.364726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462af5b-bd38-4b2f-bcd8-087154b37b7d admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.365363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462af5b-bd38-4b2f-bcd8-087154b37b7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:58.371331 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462af5b-bd38-4b2f-bcd8-087154b37b7d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:58.371699 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1065/2125] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:58.371699 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab27be-a0ff-42ee-9a6c-536e0c446a11 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:58.390614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab27be-a0ff-42ee-9a6c-536e0c446a11 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:58.397888 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d540c3-046a-4ebc-aa1f-06a2e0aa8db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:58.399379 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d540c3-046a-4ebc-aa1f-06a2e0aa8db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.399829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d540c3-046a-4ebc-aa1f-06a2e0aa8db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.400200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d540c3-046a-4ebc-aa1f-06a2e0aa8db2 admin admin] PATH_INFO: `/v3/projects/e6669485f0024c83a7154101cfa48abc/users/cf14dd15f92b474cb7608a6efb390fe9/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.408915 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1061/2126] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:23:58 2022] POST /identity/v3/users => generated 293 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:58.412135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462af5b-bd38-4b2f-bcd8-087154b37b7d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:58.429935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462af5b-bd38-4b2f-bcd8-087154b37b7d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:58.434914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f621a-154c-4a31-95b5-6c8ad7ddc885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:58.435575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f621a-154c-4a31-95b5-6c8ad7ddc885 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.435867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f621a-154c-4a31-95b5-6c8ad7ddc885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.436146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f621a-154c-4a31-95b5-6c8ad7ddc885 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.436806 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f621a-154c-4a31-95b5-6c8ad7ddc885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:58.438427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f621a-154c-4a31-95b5-6c8ad7ddc885 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:58.449230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d540c3-046a-4ebc-aa1f-06a2e0aa8db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6669485f0024c83a7154101cfa48abc, user_id=cf14dd15f92b474cb7608a6efb390fe9, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:58.451205 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1062/2127] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:58.452087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f621a-154c-4a31-95b5-6c8ad7ddc885 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:58.452860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d540c3-046a-4ebc-aa1f-06a2e0aa8db2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:58.463144 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f621a-154c-4a31-95b5-6c8ad7ddc885 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:58.468340 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a860a-77b7-4bc5-ba0c-df9d57af9d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:58.468958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a860a-77b7-4bc5-ba0c-df9d57af9d5c admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.469242 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a860a-77b7-4bc5-ba0c-df9d57af9d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.469519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a860a-77b7-4bc5-ba0c-df9d57af9d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.470219 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a860a-77b7-4bc5-ba0c-df9d57af9d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:58.471890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a860a-77b7-4bc5-ba0c-df9d57af9d5c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:58.484248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1066/2128] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:58 2022] PUT /identity/v3/projects/e6669485f0024c83a7154101cfa48abc/users/cf14dd15f92b474cb7608a6efb390fe9/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:58.485562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d540c3-046a-4ebc-aa1f-06a2e0aa8db2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:58.490017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1063/2129] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:58.495280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a860a-77b7-4bc5-ba0c-df9d57af9d5c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:58.509687 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a860a-77b7-4bc5-ba0c-df9d57af9d5c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:58.582811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d540c3-046a-4ebc-aa1f-06a2e0aa8db2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:58.587759 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949d9f3-2e83-4dce-bc5e-7926018f0327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:58.588426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949d9f3-2e83-4dce-bc5e-7926018f0327 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.588761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949d9f3-2e83-4dce-bc5e-7926018f0327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.589021 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949d9f3-2e83-4dce-bc5e-7926018f0327 admin admin] PATH_INFO: `/v3/projects/35565ce033fd466aa354f1fc7e7987a5/users/e23dec2af6ad4f9eae85d1d1c0b5f4ff/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.592375 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f7ab0-69a9-4ac8-8cf3-23a166cf835d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:58.592992 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f7ab0-69a9-4ac8-8cf3-23a166cf835d admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.593279 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f7ab0-69a9-4ac8-8cf3-23a166cf835d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.593552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f7ab0-69a9-4ac8-8cf3-23a166cf835d admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.594224 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381f7ab0-69a9-4ac8-8cf3-23a166cf835d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:58.595921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381f7ab0-69a9-4ac8-8cf3-23a166cf835d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:58.605949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949d9f3-2e83-4dce-bc5e-7926018f0327 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35565ce033fd466aa354f1fc7e7987a5, user_id=e23dec2af6ad4f9eae85d1d1c0b5f4ff, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:58.607377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1064/2130] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:58.608986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0bcf3-9a3d-4207-a15a-8f7379cfc6d7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.609277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0bcf3-9a3d-4207-a15a-8f7379cfc6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.609556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0bcf3-9a3d-4207-a15a-8f7379cfc6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.612128 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949d9f3-2e83-4dce-bc5e-7926018f0327 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:58.631031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1067/2131] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:58 2022] PUT /identity/v3/projects/35565ce033fd466aa354f1fc7e7987a5/users/e23dec2af6ad4f9eae85d1d1c0b5f4ff/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:58.631888 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949d9f3-2e83-4dce-bc5e-7926018f0327 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:58.644430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9ee0bcf3-9a3d-4207-a15a-8f7379cfc6d7 None None] MFA Rules not processed for user `cf14dd15f92b474cb7608a6efb390fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:58.645653 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9ee0bcf3-9a3d-4207-a15a-8f7379cfc6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:58.668835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:58.712733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949d9f3-2e83-4dce-bc5e-7926018f0327 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:58.717693 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68946aa3-96d0-47f4-9653-49139c06e17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:58.718425 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68946aa3-96d0-47f4-9653-49139c06e17c admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.718780 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68946aa3-96d0-47f4-9653-49139c06e17c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.719125 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68946aa3-96d0-47f4-9653-49139c06e17c admin admin] PATH_INFO: `/v3/projects/5dc5e9c9bce940e7b83cdf8f822b1895/users/9bea87129d2a47a9b583b220a0d37f41/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.730607 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1065/2132] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:58.732014 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2eaba-2ee5-402f-9ce4-18b7f7a0afac None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.732369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2eaba-2ee5-402f-9ce4-18b7f7a0afac None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.732714 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2eaba-2ee5-402f-9ce4-18b7f7a0afac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.737673 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68946aa3-96d0-47f4-9653-49139c06e17c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc5e9c9bce940e7b83cdf8f822b1895, user_id=9bea87129d2a47a9b583b220a0d37f41, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:58.742995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68946aa3-96d0-47f4-9653-49139c06e17c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:58.759667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1068/2133] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:58 2022] PUT /identity/v3/projects/5dc5e9c9bce940e7b83cdf8f822b1895/users/9bea87129d2a47a9b583b220a0d37f41/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:58.760923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68946aa3-96d0-47f4-9653-49139c06e17c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:58.770209 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-fab2eaba-2ee5-402f-9ce4-18b7f7a0afac None None] MFA Rules not processed for user `e23dec2af6ad4f9eae85d1d1c0b5f4ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:58.770590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-68946aa3-96d0-47f4-9653-49139c06e17c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:58.771520 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fab2eaba-2ee5-402f-9ce4-18b7f7a0afac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:58.794766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:58.840167 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68946aa3-96d0-47f4-9653-49139c06e17c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:58.845503 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bdc94-ca13-4492-8392-d554c3e08a79 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a889616f86346f48b7573969174ee57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:58.846819 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bdc94-ca13-4492-8392-d554c3e08a79 service glance] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.847340 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bdc94-ca13-4492-8392-d554c3e08a79 service glance] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.847900 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bdc94-ca13-4492-8392-d554c3e08a79 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.858325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1066/2134] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:58.859779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556b7b1-add7-4f75-a070-66fab12dc9ed None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:58.860067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556b7b1-add7-4f75-a070-66fab12dc9ed None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:58.860341 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556b7b1-add7-4f75-a070-66fab12dc9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:58.890784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-1556b7b1-add7-4f75-a070-66fab12dc9ed None None] MFA Rules not processed for user `9bea87129d2a47a9b583b220a0d37f41`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:58.892067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1556b7b1-add7-4f75-a070-66fab12dc9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:58.919359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44bdc94-ca13-4492-8392-d554c3e08a79 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:58.920925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44bdc94-ca13-4492-8392-d554c3e08a79 service glance] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:58.931028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1069/2135] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:58.931877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bdc94-ca13-4492-8392-d554c3e08a79 service glance] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:58.979061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1067/2136] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:58.983984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556b7b1-add7-4f75-a070-66fab12dc9ed None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:59.001839 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bdc94-ca13-4492-8392-d554c3e08a79 service glance] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:59.007155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e44851-39f2-4215-83a7-691f5bb42043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:59.008463 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e44851-39f2-4215-83a7-691f5bb42043 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:59.008749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e44851-39f2-4215-83a7-691f5bb42043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:59.009053 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e44851-39f2-4215-83a7-691f5bb42043 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:59.009694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e44851-39f2-4215-83a7-691f5bb42043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:59.011687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e44851-39f2-4215-83a7-691f5bb42043 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:59.030041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:59.053787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1070/2137] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:23:58 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:59.058321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e44851-39f2-4215-83a7-691f5bb42043 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:59.073282 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556b7b1-add7-4f75-a070-66fab12dc9ed None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:59.134612 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:59.139331 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e44851-39f2-4215-83a7-691f5bb42043 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:59.146840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285709c8-e4d4-4610-a059-175b5d5df7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:59.147495 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285709c8-e4d4-4610-a059-175b5d5df7d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:59.148639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285709c8-e4d4-4610-a059-175b5d5df7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:59.148909 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285709c8-e4d4-4610-a059-175b5d5df7d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:59.149553 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285709c8-e4d4-4610-a059-175b5d5df7d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:59.151965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285709c8-e4d4-4610-a059-175b5d5df7d1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:59.166784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d93f21-b130-447e-9f75-0014abcbdd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:59.167377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d93f21-b130-447e-9f75-0014abcbdd91 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:59.167775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d93f21-b130-447e-9f75-0014abcbdd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:59.168055 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d93f21-b130-447e-9f75-0014abcbdd91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:59.168782 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d93f21-b130-447e-9f75-0014abcbdd91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:59.171148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d93f21-b130-447e-9f75-0014abcbdd91 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:59.175896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1071/2138] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:23:59 2022] POST /identity/v3/users => generated 298 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:23:59.179425 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285709c8-e4d4-4610-a059-175b5d5df7d1 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:59.190548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285709c8-e4d4-4610-a059-175b5d5df7d1 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:59.196305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bcea4a-a900-4ac5-89c4-42d4d4377e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:59.197169 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcea4a-a900-4ac5-89c4-42d4d4377e62 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:59.197870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcea4a-a900-4ac5-89c4-42d4d4377e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:59.198715 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcea4a-a900-4ac5-89c4-42d4d4377e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:59.199593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bcea4a-a900-4ac5-89c4-42d4d4377e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:59.201323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bcea4a-a900-4ac5-89c4-42d4d4377e62 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:59.207570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1068/2139] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:23:58 2022] POST /identity/v3/projects => generated 385 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:59.209018 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d93f21-b130-447e-9f75-0014abcbdd91 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:59.213505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1072/2140] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:23:59.217546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bcea4a-a900-4ac5-89c4-42d4d4377e62 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:59.229045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bcea4a-a900-4ac5-89c4-42d4d4377e62 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:59.300376 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d93f21-b130-447e-9f75-0014abcbdd91 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:59.304512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ba550-e781-4715-9a37-5dae40ec3965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:59.305239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ba550-e781-4715-9a37-5dae40ec3965 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:59.305483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfe461-e512-40b6-81e0-ecb8db3e483f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:59.305614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ba550-e781-4715-9a37-5dae40ec3965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:59.305932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ba550-e781-4715-9a37-5dae40ec3965 admin admin] PATH_INFO: `/v3/projects/fe8b21b911d340099ebd673bedb81a7a/users/0530e753d6164457bccde81866a32fcf/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:59.306275 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfe461-e512-40b6-81e0-ecb8db3e483f admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:59.306632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfe461-e512-40b6-81e0-ecb8db3e483f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:59.307017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfe461-e512-40b6-81e0-ecb8db3e483f admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:59.307759 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfe461-e512-40b6-81e0-ecb8db3e483f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:59.310235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfe461-e512-40b6-81e0-ecb8db3e483f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:59.324048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ba550-e781-4715-9a37-5dae40ec3965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe8b21b911d340099ebd673bedb81a7a, user_id=0530e753d6164457bccde81866a32fcf, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:59.327165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ba550-e781-4715-9a37-5dae40ec3965 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:59.334133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1069/2141] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:23:59 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:23:59.338921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfe461-e512-40b6-81e0-ecb8db3e483f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:59.343368 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1073/2142] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:59 2022] PUT /identity/v3/projects/fe8b21b911d340099ebd673bedb81a7a/users/0530e753d6164457bccde81866a32fcf/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:23:59.347339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ba550-e781-4715-9a37-5dae40ec3965 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:59.350709 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfe461-e512-40b6-81e0-ecb8db3e483f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:59.409134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:59.420723 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060f5e7-2c00-49d1-9b16-77865b785991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:59.421101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ba550-e781-4715-9a37-5dae40ec3965 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:59.421583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060f5e7-2c00-49d1-9b16-77865b785991 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:59.421689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060f5e7-2c00-49d1-9b16-77865b785991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:59.421934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060f5e7-2c00-49d1-9b16-77865b785991 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:59.422583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060f5e7-2c00-49d1-9b16-77865b785991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:59.424335 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060f5e7-2c00-49d1-9b16-77865b785991 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:59.426610 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58bad6-e232-49b6-bc96-20b4e6dc4d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:59.427373 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58bad6-e232-49b6-bc96-20b4e6dc4d6e admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:59.427754 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58bad6-e232-49b6-bc96-20b4e6dc4d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:59.428102 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58bad6-e232-49b6-bc96-20b4e6dc4d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:59.428832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58bad6-e232-49b6-bc96-20b4e6dc4d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:59.430612 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58bad6-e232-49b6-bc96-20b4e6dc4d6e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:59.438937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1070/2143] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:59.439547 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060f5e7-2c00-49d1-9b16-77865b785991 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:59.448429 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1074/2144] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:23:59.449457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58bad6-e232-49b6-bc96-20b4e6dc4d6e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:59.456550 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060f5e7-2c00-49d1-9b16-77865b785991 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:59.462415 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58bad6-e232-49b6-bc96-20b4e6dc4d6e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:59.465522 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd20906-53f9-4b07-ab94-6c189a641cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:59.466204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd20906-53f9-4b07-ab94-6c189a641cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:59.466539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd20906-53f9-4b07-ab94-6c189a641cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:59.466825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd20906-53f9-4b07-ab94-6c189a641cb9 admin admin] PATH_INFO: `/v3/projects/be95dc5aa8fb4c639e257b09f9ba38ef/users/602313c9c4814ea0897916329f91c0ad/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:59.467922 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbec8c-e57e-4be9-a4c3-8880f13ce5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:59.468851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbec8c-e57e-4be9-a4c3-8880f13ce5ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:59.469235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbec8c-e57e-4be9-a4c3-8880f13ce5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:59.469606 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbec8c-e57e-4be9-a4c3-8880f13ce5ad admin admin] PATH_INFO: `/v3/projects/fe8b21b911d340099ebd673bedb81a7a/users/0530e753d6164457bccde81866a32fcf/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:59.471974 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbec8c-e57e-4be9-a4c3-8880f13ce5ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe8b21b911d340099ebd673bedb81a7a, user_id=0530e753d6164457bccde81866a32fcf, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:59.475048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbec8c-e57e-4be9-a4c3-8880f13ce5ad admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:59.487594 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd20906-53f9-4b07-ab94-6c189a641cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be95dc5aa8fb4c639e257b09f9ba38ef, user_id=602313c9c4814ea0897916329f91c0ad, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:59.491660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd20906-53f9-4b07-ab94-6c189a641cb9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:59.501425 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1075/2145] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:59 2022] PUT /identity/v3/projects/fe8b21b911d340099ebd673bedb81a7a/users/0530e753d6164457bccde81866a32fcf/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:59.503009 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b812c-8bb1-45b5-bb88-c8fd1eab07fc None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:59.503329 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b812c-8bb1-45b5-bb88-c8fd1eab07fc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:59.503623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b812c-8bb1-45b5-bb88-c8fd1eab07fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:59.530717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1071/2146] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:59 2022] PUT /identity/v3/projects/be95dc5aa8fb4c639e257b09f9ba38ef/users/602313c9c4814ea0897916329f91c0ad/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:23:59.530717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd20906-53f9-4b07-ab94-6c189a641cb9 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:59.550975 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c43b812c-8bb1-45b5-bb88-c8fd1eab07fc None None] MFA Rules not processed for user `0530e753d6164457bccde81866a32fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:59.552467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c43b812c-8bb1-45b5-bb88-c8fd1eab07fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:59.597412 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:59.613196 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd20906-53f9-4b07-ab94-6c189a641cb9 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:59.618966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55744eb8-63e5-4096-9f50-fe3a1828f017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:59.619926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55744eb8-63e5-4096-9f50-fe3a1828f017 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:59.620571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55744eb8-63e5-4096-9f50-fe3a1828f017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:59.621058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55744eb8-63e5-4096-9f50-fe3a1828f017 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:59.621950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55744eb8-63e5-4096-9f50-fe3a1828f017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:59.623804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55744eb8-63e5-4096-9f50-fe3a1828f017 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:59.635851 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1072/2147] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:23:59.639767 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55744eb8-63e5-4096-9f50-fe3a1828f017 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:59.652006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55744eb8-63e5-4096-9f50-fe3a1828f017 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:59.652998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1076/2148] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:59 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:23:59.666160 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b812c-8bb1-45b5-bb88-c8fd1eab07fc None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:59.670429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f23bf8-4708-443a-a99c-2c14d3a24404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:59.671125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f23bf8-4708-443a-a99c-2c14d3a24404 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:59.671928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f23bf8-4708-443a-a99c-2c14d3a24404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:59.672211 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f23bf8-4708-443a-a99c-2c14d3a24404 admin admin] PATH_INFO: `/v3/projects/be95dc5aa8fb4c639e257b09f9ba38ef/users/602313c9c4814ea0897916329f91c0ad/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:59.674433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f23bf8-4708-443a-a99c-2c14d3a24404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be95dc5aa8fb4c639e257b09f9ba38ef, user_id=602313c9c4814ea0897916329f91c0ad, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:59.677424 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f23bf8-4708-443a-a99c-2c14d3a24404 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:59.695410 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1073/2149] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:23:59 2022] PUT /identity/v3/projects/be95dc5aa8fb4c639e257b09f9ba38ef/users/602313c9c4814ea0897916329f91c0ad/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:23:59.710782 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1eb3d-017e-4d48-9aea-1305207c9c49 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:59.712115 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1eb3d-017e-4d48-9aea-1305207c9c49 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:59.712682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1eb3d-017e-4d48-9aea-1305207c9c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:59.737164 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b812c-8bb1-45b5-bb88-c8fd1eab07fc None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:59.757752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-87d1eb3d-017e-4d48-9aea-1305207c9c49 None None] MFA Rules not processed for user `602313c9c4814ea0897916329f91c0ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:23:59.759152 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-87d1eb3d-017e-4d48-9aea-1305207c9c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:59.802341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaec6c4-c21a-43fb-9520-6aee77ba7845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:59.802981 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaec6c4-c21a-43fb-9520-6aee77ba7845 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:59.803296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaec6c4-c21a-43fb-9520-6aee77ba7845 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:59.803601 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaec6c4-c21a-43fb-9520-6aee77ba7845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:59.875087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaec6c4-c21a-43fb-9520-6aee77ba7845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:59.876865 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaec6c4-c21a-43fb-9520-6aee77ba7845 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:23:59.895904 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1074/2150] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:23:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:23:59.896004 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1077/2151] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:23:59.903074 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1eb3d-017e-4d48-9aea-1305207c9c49 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:23:59.904742 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-87d1eb3d-017e-4d48-9aea-1305207c9c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:23:59.919896 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1eb3d-017e-4d48-9aea-1305207c9c49 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:23:59.924148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23999d8-d65b-4f9a-98ac-49451516661f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:23:59.925032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23999d8-d65b-4f9a-98ac-49451516661f service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:23:59.925410 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23999d8-d65b-4f9a-98ac-49451516661f service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:23:59.925744 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23999d8-d65b-4f9a-98ac-49451516661f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:23:59.962517 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:23:59.991343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23999d8-d65b-4f9a-98ac-49451516661f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:23:59.993449 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23999d8-d65b-4f9a-98ac-49451516661f service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:00.002652 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1075/2152] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:23:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:02.736410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaec6c4-c21a-43fb-9520-6aee77ba7845 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:02.745474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaec6c4-c21a-43fb-9520-6aee77ba7845 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:02.749594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7fed5-2fe7-4b5e-a1a8-9e364a868526 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:02.750222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7fed5-2fe7-4b5e-a1a8-9e364a868526 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:02.750509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7fed5-2fe7-4b5e-a1a8-9e364a868526 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:02.751559 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7fed5-2fe7-4b5e-a1a8-9e364a868526 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:02.788389 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:02.816716 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c7fed5-2fe7-4b5e-a1a8-9e364a868526 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:02.818209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c7fed5-2fe7-4b5e-a1a8-9e364a868526 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:02.832063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1078/2153] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:05.207707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23999d8-d65b-4f9a-98ac-49451516661f service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:05.214570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23999d8-d65b-4f9a-98ac-49451516661f service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:05.218114 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f72ed-53bb-4c73-9dbe-3b131bc763e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:05.218709 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f72ed-53bb-4c73-9dbe-3b131bc763e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:05.218985 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f72ed-53bb-4c73-9dbe-3b131bc763e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:05.219253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f72ed-53bb-4c73-9dbe-3b131bc763e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:05.282200 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f72ed-53bb-4c73-9dbe-3b131bc763e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:05.283936 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f72ed-53bb-4c73-9dbe-3b131bc763e2 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:05.293289 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1076/2154] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:10.094649 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7fed5-2fe7-4b5e-a1a8-9e364a868526 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:10.105483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7fed5-2fe7-4b5e-a1a8-9e364a868526 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:10.109213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5c4dc-8471-4626-bf79-0b27958990af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:10.109807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5c4dc-8471-4626-bf79-0b27958990af service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:10.110084 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5c4dc-8471-4626-bf79-0b27958990af service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:10.110361 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5c4dc-8471-4626-bf79-0b27958990af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:10.171293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5c4dc-8471-4626-bf79-0b27958990af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:10.172759 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5c4dc-8471-4626-bf79-0b27958990af service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:10.182002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1079/2155] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:12.952612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f72ed-53bb-4c73-9dbe-3b131bc763e2 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:12.961746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f72ed-53bb-4c73-9dbe-3b131bc763e2 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:12.966972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129737b7-82f1-4f19-88bd-81228c912c87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:12.966972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129737b7-82f1-4f19-88bd-81228c912c87 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:12.966972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129737b7-82f1-4f19-88bd-81228c912c87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:12.966972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129737b7-82f1-4f19-88bd-81228c912c87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:12.976947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129737b7-82f1-4f19-88bd-81228c912c87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:12.978925 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129737b7-82f1-4f19-88bd-81228c912c87 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:12.988427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1077/2156] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:20.178274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=39 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:22.984167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=49 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:30.481338 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5c4dc-8471-4626-bf79-0b27958990af service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:30.548335 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5c4dc-8471-4626-bf79-0b27958990af service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:30.554824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89926d84-bb75-4da6-9761-06e733a4487c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:30.555498 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89926d84-bb75-4da6-9761-06e733a4487c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:30.555801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89926d84-bb75-4da6-9761-06e733a4487c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:30.556070 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89926d84-bb75-4da6-9761-06e733a4487c admin admin] PATH_INFO: `/v3/users/89f04190e02b446a94a7cf81ab1d5b16` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:30.557234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89926d84-bb75-4da6-9761-06e733a4487c admin admin] RBAC: Authorizing `identity:delete_user(user_id=89f04190e02b446a94a7cf81ab1d5b16)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:30.559340 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89926d84-bb75-4da6-9761-06e733a4487c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:30.611462 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-89926d84-bb75-4da6-9761-06e733a4487c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89f04190e02b446a94a7cf81ab1d5b16'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:30.619778 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-89926d84-bb75-4da6-9761-06e733a4487c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89f04190e02b446a94a7cf81ab1d5b16'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:30.620524 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-89926d84-bb75-4da6-9761-06e733a4487c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89f04190e02b446a94a7cf81ab1d5b16'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:30.621199 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-89926d84-bb75-4da6-9761-06e733a4487c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89f04190e02b446a94a7cf81ab1d5b16'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:30.629302 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-89926d84-bb75-4da6-9761-06e733a4487c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89f04190e02b446a94a7cf81ab1d5b16'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:30.648466 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1080/2157] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:24:30 2022] DELETE /identity/v3/users/89f04190e02b446a94a7cf81ab1d5b16 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:30.652802 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982aace3-1ca2-4f41-a86f-60120851086a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:30.653304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982aace3-1ca2-4f41-a86f-60120851086a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:30.653396 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982aace3-1ca2-4f41-a86f-60120851086a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:30.674249 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-982aace3-1ca2-4f41-a86f-60120851086a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:30.675681 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-982aace3-1ca2-4f41-a86f-60120851086a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:30.688620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89926d84-bb75-4da6-9761-06e733a4487c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:30.690016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-89926d84-bb75-4da6-9761-06e733a4487c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:30.747053 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1078/2158] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:30.760996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982aace3-1ca2-4f41-a86f-60120851086a None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:30.762746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-982aace3-1ca2-4f41-a86f-60120851086a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:30.785274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89926d84-bb75-4da6-9761-06e733a4487c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:30.791755 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe6520-b44c-4384-a045-12bb914f3c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:30.792613 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe6520-b44c-4384-a045-12bb914f3c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:30.793058 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe6520-b44c-4384-a045-12bb914f3c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:30.793144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe6520-b44c-4384-a045-12bb914f3c3f admin admin] PATH_INFO: `/v3/users/9bdc859ae3e44327b52d1dca32279c05` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:30.794331 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe6520-b44c-4384-a045-12bb914f3c3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bdc859ae3e44327b52d1dca32279c05)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:30.796343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe6520-b44c-4384-a045-12bb914f3c3f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:30.831764 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:30.848263 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982aace3-1ca2-4f41-a86f-60120851086a None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:30.853300 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610eb74-950c-4a96-9869-864ef9c34aff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:30.854047 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610eb74-950c-4a96-9869-864ef9c34aff service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:30.854333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610eb74-950c-4a96-9869-864ef9c34aff service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:30.854600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610eb74-950c-4a96-9869-864ef9c34aff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:30.864299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-23fe6520-b44c-4384-a045-12bb914f3c3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bdc859ae3e44327b52d1dca32279c05'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:30.865159 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610eb74-950c-4a96-9869-864ef9c34aff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:30.866827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610eb74-950c-4a96-9869-864ef9c34aff service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:30.872776 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-23fe6520-b44c-4384-a045-12bb914f3c3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bdc859ae3e44327b52d1dca32279c05'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:30.873484 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-23fe6520-b44c-4384-a045-12bb914f3c3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bdc859ae3e44327b52d1dca32279c05'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:30.874120 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-23fe6520-b44c-4384-a045-12bb914f3c3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bdc859ae3e44327b52d1dca32279c05'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:30.882927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-23fe6520-b44c-4384-a045-12bb914f3c3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bdc859ae3e44327b52d1dca32279c05'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:30.883695 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1079/2159] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:30.885298 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610eb74-950c-4a96-9869-864ef9c34aff service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:30.886589 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8610eb74-950c-4a96-9869-864ef9c34aff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:30.899736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610eb74-950c-4a96-9869-864ef9c34aff service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:30.902827 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1081/2160] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:24:30 2022] DELETE /identity/v3/users/9bdc859ae3e44327b52d1dca32279c05 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:30.904677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fffd72-32b9-4eb1-8026-6561fd0d1e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:30.905322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fffd72-32b9-4eb1-8026-6561fd0d1e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:30.905841 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fffd72-32b9-4eb1-8026-6561fd0d1e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:30.906120 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fffd72-32b9-4eb1-8026-6561fd0d1e73 admin admin] PATH_INFO: `/v3/users/3dc1230116cb46d9a3a4110cabf6caf5` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:30.907091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa871cc4-c4b0-48fb-8679-91b106739f70 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:30.907299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fffd72-32b9-4eb1-8026-6561fd0d1e73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dc1230116cb46d9a3a4110cabf6caf5)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:30.907397 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa871cc4-c4b0-48fb-8679-91b106739f70 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:30.907920 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa871cc4-c4b0-48fb-8679-91b106739f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:30.909486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fffd72-32b9-4eb1-8026-6561fd0d1e73 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:30.927757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fa871cc4-c4b0-48fb-8679-91b106739f70 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:30.928894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fa871cc4-c4b0-48fb-8679-91b106739f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:30.964827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a7fffd72-32b9-4eb1-8026-6561fd0d1e73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dc1230116cb46d9a3a4110cabf6caf5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:30.975130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a7fffd72-32b9-4eb1-8026-6561fd0d1e73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dc1230116cb46d9a3a4110cabf6caf5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:30.976165 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a7fffd72-32b9-4eb1-8026-6561fd0d1e73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dc1230116cb46d9a3a4110cabf6caf5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:30.976548 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a7fffd72-32b9-4eb1-8026-6561fd0d1e73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dc1230116cb46d9a3a4110cabf6caf5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:30.986946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a7fffd72-32b9-4eb1-8026-6561fd0d1e73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dc1230116cb46d9a3a4110cabf6caf5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.018910 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1082/2161] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:24:31.019627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1080/2162] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:24:30 2022] DELETE /identity/v3/users/3dc1230116cb46d9a3a4110cabf6caf5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:31.025671 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e45a0-7ca7-4f29-8abd-d4c80fa60b9f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:31.026039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e45a0-7ca7-4f29-8abd-d4c80fa60b9f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:31.026387 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e45a0-7ca7-4f29-8abd-d4c80fa60b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:31.031090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fffd72-32b9-4eb1-8026-6561fd0d1e73 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:31.033778 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a7fffd72-32b9-4eb1-8026-6561fd0d1e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:31.050836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-956e45a0-7ca7-4f29-8abd-d4c80fa60b9f None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:31.050836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-956e45a0-7ca7-4f29-8abd-d4c80fa60b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:31.073781 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:31.139031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fffd72-32b9-4eb1-8026-6561fd0d1e73 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:31.141733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1083/2163] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:31.144679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c3056-6bc2-4807-8740-d590e09c5ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:31.145574 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c3056-6bc2-4807-8740-d590e09c5ffe service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:31.145865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c3056-6bc2-4807-8740-d590e09c5ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:31.146661 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c3056-6bc2-4807-8740-d590e09c5ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:31.166732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e45a0-7ca7-4f29-8abd-d4c80fa60b9f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:31.174844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c3056-6bc2-4807-8740-d590e09c5ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:31.176684 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c3056-6bc2-4807-8740-d590e09c5ffe service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:31.188775 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e45a0-7ca7-4f29-8abd-d4c80fa60b9f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:31.195886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1081/2164] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:24:31.201437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e396f-9c1b-41b1-9bb7-ddcb09282af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:31.202072 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e396f-9c1b-41b1-9bb7-ddcb09282af2 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:31.202353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e396f-9c1b-41b1-9bb7-ddcb09282af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:31.202761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e396f-9c1b-41b1-9bb7-ddcb09282af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:31.222691 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e396f-9c1b-41b1-9bb7-ddcb09282af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:31.225313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e396f-9c1b-41b1-9bb7-ddcb09282af2 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:31.242733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1084/2165] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:31.350173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c3056-6bc2-4807-8740-d590e09c5ffe service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:31.354093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e64c3056-6bc2-4807-8740-d590e09c5ffe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:31.367410 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c3056-6bc2-4807-8740-d590e09c5ffe service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:31.372959 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336125ce-8146-4923-b521-b7b05402e834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:31.373642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336125ce-8146-4923-b521-b7b05402e834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:31.374284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336125ce-8146-4923-b521-b7b05402e834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:31.374284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336125ce-8146-4923-b521-b7b05402e834 admin admin] PATH_INFO: `/v3/projects/9bc37605c3444676a9783bcac463c2d5` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:31.375632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336125ce-8146-4923-b521-b7b05402e834 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bc37605c3444676a9783bcac463c2d5)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:31.378070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336125ce-8146-4923-b521-b7b05402e834 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:31.444107 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-336125ce-8146-4923-b521-b7b05402e834 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bc37605c3444676a9783bcac463c2d5', 'request_id': 'req-336125ce-8146-4923-b521-b7b05402e834'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.455550 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-336125ce-8146-4923-b521-b7b05402e834 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bc37605c3444676a9783bcac463c2d5', 'request_id': 'req-336125ce-8146-4923-b521-b7b05402e834'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.470023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-336125ce-8146-4923-b521-b7b05402e834 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bc37605c3444676a9783bcac463c2d5', 'request_id': 'req-336125ce-8146-4923-b521-b7b05402e834'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.485322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-336125ce-8146-4923-b521-b7b05402e834 admin admin] The token cache is being invalidate because project 9bc37605c3444676a9783bcac463c2d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:24:31.485787 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-336125ce-8146-4923-b521-b7b05402e834 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.486608 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-336125ce-8146-4923-b521-b7b05402e834 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.488455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1082/2166] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:24:31 2022] DELETE /identity/v3/projects/9bc37605c3444676a9783bcac463c2d5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:31.493307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e396f-9c1b-41b1-9bb7-ddcb09282af2 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:31.509109 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e396f-9c1b-41b1-9bb7-ddcb09282af2 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:31.546327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:31.585207 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e83d0-b3f5-4305-ad65-51b6f47943eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:31.585987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e83d0-b3f5-4305-ad65-51b6f47943eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:31.586337 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e83d0-b3f5-4305-ad65-51b6f47943eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:31.586680 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e83d0-b3f5-4305-ad65-51b6f47943eb admin admin] PATH_INFO: `/v3/users/e23dec2af6ad4f9eae85d1d1c0b5f4ff` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:31.587951 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e83d0-b3f5-4305-ad65-51b6f47943eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e23dec2af6ad4f9eae85d1d1c0b5f4ff)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:31.590095 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e83d0-b3f5-4305-ad65-51b6f47943eb admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:31.644296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a1e83d0-b3f5-4305-ad65-51b6f47943eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e23dec2af6ad4f9eae85d1d1c0b5f4ff'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.651847 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a1e83d0-b3f5-4305-ad65-51b6f47943eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e23dec2af6ad4f9eae85d1d1c0b5f4ff'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.652533 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a1e83d0-b3f5-4305-ad65-51b6f47943eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e23dec2af6ad4f9eae85d1d1c0b5f4ff'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.653146 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a1e83d0-b3f5-4305-ad65-51b6f47943eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e23dec2af6ad4f9eae85d1d1c0b5f4ff'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.661146 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4a1e83d0-b3f5-4305-ad65-51b6f47943eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e23dec2af6ad4f9eae85d1d1c0b5f4ff'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.679454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1085/2167] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:24:31 2022] DELETE /identity/v3/users/e23dec2af6ad4f9eae85d1d1c0b5f4ff => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:31.755465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336125ce-8146-4923-b521-b7b05402e834 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:31.757682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-336125ce-8146-4923-b521-b7b05402e834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:31.763238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e83d0-b3f5-4305-ad65-51b6f47943eb admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:31.765353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4a1e83d0-b3f5-4305-ad65-51b6f47943eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:31.782438 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336125ce-8146-4923-b521-b7b05402e834 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:31.857349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e83d0-b3f5-4305-ad65-51b6f47943eb admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:31.862920 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f2d4a-65a4-43a4-9684-26c8e955c273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:31.863615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f2d4a-65a4-43a4-9684-26c8e955c273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:31.863899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f2d4a-65a4-43a4-9684-26c8e955c273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:31.864175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f2d4a-65a4-43a4-9684-26c8e955c273 admin admin] PATH_INFO: `/v3/projects/f378e738b9c144799bf5f4b875048455` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:31.868607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f2d4a-65a4-43a4-9684-26c8e955c273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f378e738b9c144799bf5f4b875048455)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:31.868910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:31.869006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f2d4a-65a4-43a4-9684-26c8e955c273 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:31.869092 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:31.869092 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:31.869092 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d admin admin] PATH_INFO: `/v3/projects/468a5076e9994c5bb5691cc1146b1c22` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:31.870852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d admin admin] RBAC: Authorizing `identity:delete_project(project_id=468a5076e9994c5bb5691cc1146b1c22)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:31.872820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:31.935832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:31.938480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3b5f2d4a-65a4-43a4-9684-26c8e955c273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f378e738b9c144799bf5f4b875048455', 'request_id': 'req-3b5f2d4a-65a4-43a4-9684-26c8e955c273'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.939348 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:31.945251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '468a5076e9994c5bb5691cc1146b1c22', 'request_id': 'req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.953148 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3b5f2d4a-65a4-43a4-9684-26c8e955c273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f378e738b9c144799bf5f4b875048455', 'request_id': 'req-3b5f2d4a-65a4-43a4-9684-26c8e955c273'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.956363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '468a5076e9994c5bb5691cc1146b1c22', 'request_id': 'req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.967696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '468a5076e9994c5bb5691cc1146b1c22', 'request_id': 'req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.968276 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3b5f2d4a-65a4-43a4-9684-26c8e955c273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f378e738b9c144799bf5f4b875048455', 'request_id': 'req-3b5f2d4a-65a4-43a4-9684-26c8e955c273'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.982958 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3b5f2d4a-65a4-43a4-9684-26c8e955c273 admin admin] The token cache is being invalidate because project f378e738b9c144799bf5f4b875048455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:24:31.983279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3b5f2d4a-65a4-43a4-9684-26c8e955c273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.988506 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3b5f2d4a-65a4-43a4-9684-26c8e955c273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:31.990241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1086/2168] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:24:31 2022] DELETE /identity/v3/projects/f378e738b9c144799bf5f4b875048455 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:24:31.997723 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d admin admin] The token cache is being invalidate because project 468a5076e9994c5bb5691cc1146b1c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:24:31.998023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.001078 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.002130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1083/2169] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:24:31 2022] DELETE /identity/v3/projects/468a5076e9994c5bb5691cc1146b1c22 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:32.006328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f2d4a-65a4-43a4-9684-26c8e955c273 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:32.008591 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3b5f2d4a-65a4-43a4-9684-26c8e955c273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:32.051371 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:32.053883 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:32.144530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f2d4a-65a4-43a4-9684-26c8e955c273 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:32.152675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1d03c-4bac-4d3d-acf2-0e8a29fd2a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:32.155101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1d03c-4bac-4d3d-acf2-0e8a29fd2a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:32.155879 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1d03c-4bac-4d3d-acf2-0e8a29fd2a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:32.156874 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1d03c-4bac-4d3d-acf2-0e8a29fd2a6b admin admin] PATH_INFO: `/v3/users/9bea87129d2a47a9b583b220a0d37f41` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:32.158864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1d03c-4bac-4d3d-acf2-0e8a29fd2a6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bea87129d2a47a9b583b220a0d37f41)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:32.162124 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1d03c-4bac-4d3d-acf2-0e8a29fd2a6b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:32.176835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7b0a7-c5fa-4ba0-8ccb-1ae35619f43d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:32.182603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c534b9-4996-43b8-9fe9-cf56d253deb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:32.183263 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c534b9-4996-43b8-9fe9-cf56d253deb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:32.183589 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c534b9-4996-43b8-9fe9-cf56d253deb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:32.183868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c534b9-4996-43b8-9fe9-cf56d253deb3 admin admin] PATH_INFO: `/v3/projects/35565ce033fd466aa354f1fc7e7987a5` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:32.185064 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c534b9-4996-43b8-9fe9-cf56d253deb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35565ce033fd466aa354f1fc7e7987a5)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:32.186999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c534b9-4996-43b8-9fe9-cf56d253deb3 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:32.225718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1cb1d03c-4bac-4d3d-acf2-0e8a29fd2a6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bea87129d2a47a9b583b220a0d37f41'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.233474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1cb1d03c-4bac-4d3d-acf2-0e8a29fd2a6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bea87129d2a47a9b583b220a0d37f41'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.234181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1cb1d03c-4bac-4d3d-acf2-0e8a29fd2a6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bea87129d2a47a9b583b220a0d37f41'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.234817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1cb1d03c-4bac-4d3d-acf2-0e8a29fd2a6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bea87129d2a47a9b583b220a0d37f41'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.243231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-51c534b9-4996-43b8-9fe9-cf56d253deb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35565ce033fd466aa354f1fc7e7987a5', 'request_id': 'req-51c534b9-4996-43b8-9fe9-cf56d253deb3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.249323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1cb1d03c-4bac-4d3d-acf2-0e8a29fd2a6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bea87129d2a47a9b583b220a0d37f41'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.257849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-51c534b9-4996-43b8-9fe9-cf56d253deb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35565ce033fd466aa354f1fc7e7987a5', 'request_id': 'req-51c534b9-4996-43b8-9fe9-cf56d253deb3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.266287 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-51c534b9-4996-43b8-9fe9-cf56d253deb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35565ce033fd466aa354f1fc7e7987a5', 'request_id': 'req-51c534b9-4996-43b8-9fe9-cf56d253deb3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.274158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1087/2170] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:24:32 2022] DELETE /identity/v3/users/9bea87129d2a47a9b583b220a0d37f41 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:32.275099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1d03c-4bac-4d3d-acf2-0e8a29fd2a6b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:32.278487 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-51c534b9-4996-43b8-9fe9-cf56d253deb3 admin admin] The token cache is being invalidate because project 35565ce033fd466aa354f1fc7e7987a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:24:32.278911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-51c534b9-4996-43b8-9fe9-cf56d253deb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.279618 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-51c534b9-4996-43b8-9fe9-cf56d253deb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.283576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1084/2171] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:24:32 2022] DELETE /identity/v3/projects/35565ce033fd466aa354f1fc7e7987a5 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:32.285701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c534b9-4996-43b8-9fe9-cf56d253deb3 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:32.293387 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1cb1d03c-4bac-4d3d-acf2-0e8a29fd2a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:32.298627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-51c534b9-4996-43b8-9fe9-cf56d253deb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:32.387260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:32.389306 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c534b9-4996-43b8-9fe9-cf56d253deb3 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:32.396313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1d03c-4bac-4d3d-acf2-0e8a29fd2a6b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:32.402589 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dab7ae-5630-4eca-b784-ff62ffc4ef03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:32.403241 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dab7ae-5630-4eca-b784-ff62ffc4ef03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:32.403567 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dab7ae-5630-4eca-b784-ff62ffc4ef03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:32.403839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dab7ae-5630-4eca-b784-ff62ffc4ef03 admin admin] PATH_INFO: `/v3/users/0530e753d6164457bccde81866a32fcf` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:32.404996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dab7ae-5630-4eca-b784-ff62ffc4ef03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0530e753d6164457bccde81866a32fcf)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:32.406903 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dab7ae-5630-4eca-b784-ff62ffc4ef03 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:32.410192 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:32.413312 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c52a6-89fe-43a9-95cb-931840976a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:32.414063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c52a6-89fe-43a9-95cb-931840976a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:32.414657 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c52a6-89fe-43a9-95cb-931840976a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:32.415091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c52a6-89fe-43a9-95cb-931840976a97 admin admin] PATH_INFO: `/v3/users/e911cf941fbb47dea9bb0946a446177c` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:32.416489 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c52a6-89fe-43a9-95cb-931840976a97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e911cf941fbb47dea9bb0946a446177c)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:32.420798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c52a6-89fe-43a9-95cb-931840976a97 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:32.491000 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-52dab7ae-5630-4eca-b784-ff62ffc4ef03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0530e753d6164457bccde81866a32fcf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.499116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5d2c52a6-89fe-43a9-95cb-931840976a97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e911cf941fbb47dea9bb0946a446177c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.501992 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-52dab7ae-5630-4eca-b784-ff62ffc4ef03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0530e753d6164457bccde81866a32fcf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.502710 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-52dab7ae-5630-4eca-b784-ff62ffc4ef03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0530e753d6164457bccde81866a32fcf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.503759 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-52dab7ae-5630-4eca-b784-ff62ffc4ef03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0530e753d6164457bccde81866a32fcf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.506460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5d2c52a6-89fe-43a9-95cb-931840976a97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e911cf941fbb47dea9bb0946a446177c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.507800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5d2c52a6-89fe-43a9-95cb-931840976a97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e911cf941fbb47dea9bb0946a446177c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.508436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5d2c52a6-89fe-43a9-95cb-931840976a97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e911cf941fbb47dea9bb0946a446177c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.513381 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-52dab7ae-5630-4eca-b784-ff62ffc4ef03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0530e753d6164457bccde81866a32fcf'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.517077 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5d2c52a6-89fe-43a9-95cb-931840976a97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e911cf941fbb47dea9bb0946a446177c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.535907 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1085/2172] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:24:32 2022] DELETE /identity/v3/users/0530e753d6164457bccde81866a32fcf => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:24:32.539642 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1088/2173] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:24:32 2022] DELETE /identity/v3/users/e911cf941fbb47dea9bb0946a446177c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:32.543893 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66337d9-7aa0-48b3-b832-34c802765847 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:32.544307 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66337d9-7aa0-48b3-b832-34c802765847 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:32.544586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66337d9-7aa0-48b3-b832-34c802765847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:32.565041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c66337d9-7aa0-48b3-b832-34c802765847 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:32.566273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c66337d9-7aa0-48b3-b832-34c802765847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:32.618706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c52a6-89fe-43a9-95cb-931840976a97 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:32.620960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5d2c52a6-89fe-43a9-95cb-931840976a97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:32.668011 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1086/2174] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:24:32.684334 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66337d9-7aa0-48b3-b832-34c802765847 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:32.687625 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c66337d9-7aa0-48b3-b832-34c802765847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:32.751449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c52a6-89fe-43a9-95cb-931840976a97 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:32.758923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e662c-7bf4-43ba-b275-1a4af5bf26ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:32.759757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e662c-7bf4-43ba-b275-1a4af5bf26ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:32.760115 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e662c-7bf4-43ba-b275-1a4af5bf26ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:32.760466 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e662c-7bf4-43ba-b275-1a4af5bf26ec admin admin] PATH_INFO: `/v3/projects/5dc5e9c9bce940e7b83cdf8f822b1895` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:32.761785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e662c-7bf4-43ba-b275-1a4af5bf26ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dc5e9c9bce940e7b83cdf8f822b1895)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:32.765357 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e662c-7bf4-43ba-b275-1a4af5bf26ec admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:32.779793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66337d9-7aa0-48b3-b832-34c802765847 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:32.784407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6feeade-e082-4259-83c8-8195685731d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:32.785087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6feeade-e082-4259-83c8-8195685731d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:32.785374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6feeade-e082-4259-83c8-8195685731d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:32.785816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6feeade-e082-4259-83c8-8195685731d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:32.800253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6feeade-e082-4259-83c8-8195685731d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:32.802773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6feeade-e082-4259-83c8-8195685731d3 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:32.819472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1087/2175] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:32.823249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-569e662c-7bf4-43ba-b275-1a4af5bf26ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dc5e9c9bce940e7b83cdf8f822b1895', 'request_id': 'req-569e662c-7bf4-43ba-b275-1a4af5bf26ec'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.833224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-569e662c-7bf4-43ba-b275-1a4af5bf26ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dc5e9c9bce940e7b83cdf8f822b1895', 'request_id': 'req-569e662c-7bf4-43ba-b275-1a4af5bf26ec'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.840837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-569e662c-7bf4-43ba-b275-1a4af5bf26ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dc5e9c9bce940e7b83cdf8f822b1895', 'request_id': 'req-569e662c-7bf4-43ba-b275-1a4af5bf26ec'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.852407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-569e662c-7bf4-43ba-b275-1a4af5bf26ec admin admin] The token cache is being invalidate because project 5dc5e9c9bce940e7b83cdf8f822b1895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:24:32.852701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-569e662c-7bf4-43ba-b275-1a4af5bf26ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.853506 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-569e662c-7bf4-43ba-b275-1a4af5bf26ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:32.855259 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1089/2176] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:24:32 2022] DELETE /identity/v3/projects/5dc5e9c9bce940e7b83cdf8f822b1895 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:24:32.859151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6feeade-e082-4259-83c8-8195685731d3 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:32.869946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6feeade-e082-4259-83c8-8195685731d3 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:32.893073 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:32.959138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd00502-0c7d-4446-b63f-118049efef56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:32.960057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd00502-0c7d-4446-b63f-118049efef56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:32.960527 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd00502-0c7d-4446-b63f-118049efef56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:32.960886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd00502-0c7d-4446-b63f-118049efef56 admin admin] PATH_INFO: `/v3/users/602313c9c4814ea0897916329f91c0ad` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:32.962881 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd00502-0c7d-4446-b63f-118049efef56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=602313c9c4814ea0897916329f91c0ad)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:32.964577 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd00502-0c7d-4446-b63f-118049efef56 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:33.023990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ffd00502-0c7d-4446-b63f-118049efef56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '602313c9c4814ea0897916329f91c0ad'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.032033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ffd00502-0c7d-4446-b63f-118049efef56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '602313c9c4814ea0897916329f91c0ad'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.033044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ffd00502-0c7d-4446-b63f-118049efef56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '602313c9c4814ea0897916329f91c0ad'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.033834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ffd00502-0c7d-4446-b63f-118049efef56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '602313c9c4814ea0897916329f91c0ad'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.045392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ffd00502-0c7d-4446-b63f-118049efef56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '602313c9c4814ea0897916329f91c0ad'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.075600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1088/2177] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:24:32 2022] DELETE /identity/v3/users/602313c9c4814ea0897916329f91c0ad => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:33.093859 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e662c-7bf4-43ba-b275-1a4af5bf26ec admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:33.095324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-569e662c-7bf4-43ba-b275-1a4af5bf26ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:33.104546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:33.108794 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e662c-7bf4-43ba-b275-1a4af5bf26ec admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:33.193496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118d4e0-c774-4857-b337-202016afff28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:33.194162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118d4e0-c774-4857-b337-202016afff28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:33.194446 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118d4e0-c774-4857-b337-202016afff28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:33.195678 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118d4e0-c774-4857-b337-202016afff28 admin admin] PATH_INFO: `/v3/projects/fe8b21b911d340099ebd673bedb81a7a` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:33.198443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118d4e0-c774-4857-b337-202016afff28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe8b21b911d340099ebd673bedb81a7a)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:33.200538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118d4e0-c774-4857-b337-202016afff28 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:33.284584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8118d4e0-c774-4857-b337-202016afff28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe8b21b911d340099ebd673bedb81a7a', 'request_id': 'req-8118d4e0-c774-4857-b337-202016afff28'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.295077 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8118d4e0-c774-4857-b337-202016afff28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe8b21b911d340099ebd673bedb81a7a', 'request_id': 'req-8118d4e0-c774-4857-b337-202016afff28'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.304638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8118d4e0-c774-4857-b337-202016afff28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe8b21b911d340099ebd673bedb81a7a', 'request_id': 'req-8118d4e0-c774-4857-b337-202016afff28'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.320131 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8118d4e0-c774-4857-b337-202016afff28 admin admin] The token cache is being invalidate because project fe8b21b911d340099ebd673bedb81a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:24:33.320430 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8118d4e0-c774-4857-b337-202016afff28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.321143 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-8118d4e0-c774-4857-b337-202016afff28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.323104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1090/2178] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:24:33 2022] DELETE /identity/v3/projects/fe8b21b911d340099ebd673bedb81a7a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:24:33.328143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f042c7-21df-4225-bb23-0830d9edebe4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:33.328731 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f042c7-21df-4225-bb23-0830d9edebe4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:33.329116 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f042c7-21df-4225-bb23-0830d9edebe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:33.352747 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e5f042c7-21df-4225-bb23-0830d9edebe4 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:33.354123 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e5f042c7-21df-4225-bb23-0830d9edebe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:33.421351 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118d4e0-c774-4857-b337-202016afff28 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:33.422873 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8118d4e0-c774-4857-b337-202016afff28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:33.434681 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1089/2179] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:33.442083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23541fd-b77e-4051-866d-8070326fae58 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:33.442368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23541fd-b77e-4051-866d-8070326fae58 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:33.442716 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23541fd-b77e-4051-866d-8070326fae58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:33.479638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-e23541fd-b77e-4051-866d-8070326fae58 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:33.480912 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e23541fd-b77e-4051-866d-8070326fae58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:33.489823 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1090/2180] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:24:33.494327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23541fd-b77e-4051-866d-8070326fae58 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:33.503740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118d4e0-c774-4857-b337-202016afff28 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:33.504256 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23541fd-b77e-4051-866d-8070326fae58 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:33.510419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a397b7-c4f2-4220-9ba9-c2f208d4ca12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:33.511230 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a397b7-c4f2-4220-9ba9-c2f208d4ca12 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:33.511538 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a397b7-c4f2-4220-9ba9-c2f208d4ca12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:33.511717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f727a7-8751-4f44-9414-608a347e5cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:33.511850 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a397b7-c4f2-4220-9ba9-c2f208d4ca12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:33.512372 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f727a7-8751-4f44-9414-608a347e5cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:33.512481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a397b7-c4f2-4220-9ba9-c2f208d4ca12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:24:33.512655 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f727a7-8751-4f44-9414-608a347e5cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:33.512836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a397b7-c4f2-4220-9ba9-c2f208d4ca12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:33.512928 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f727a7-8751-4f44-9414-608a347e5cb4 admin admin] PATH_INFO: `/v3/projects/be95dc5aa8fb4c639e257b09f9ba38ef` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:33.514066 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f727a7-8751-4f44-9414-608a347e5cb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be95dc5aa8fb4c639e257b09f9ba38ef)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:33.515009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a397b7-c4f2-4220-9ba9-c2f208d4ca12 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:33.516124 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f727a7-8751-4f44-9414-608a347e5cb4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:33.530595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1091/2181] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:24:33.531755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a397b7-c4f2-4220-9ba9-c2f208d4ca12 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:33.544284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a397b7-c4f2-4220-9ba9-c2f208d4ca12 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:33.551196 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badce16-666d-4e43-afc1-c0656464cc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:33.551859 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badce16-666d-4e43-afc1-c0656464cc5e admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:33.552142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badce16-666d-4e43-afc1-c0656464cc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:33.552467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badce16-666d-4e43-afc1-c0656464cc5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:33.553102 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badce16-666d-4e43-afc1-c0656464cc5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:33.556866 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badce16-666d-4e43-afc1-c0656464cc5e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:33.574800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:33.584480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-38f727a7-8751-4f44-9414-608a347e5cb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be95dc5aa8fb4c639e257b09f9ba38ef', 'request_id': 'req-38f727a7-8751-4f44-9414-608a347e5cb4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.592117 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1092/2182] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:24:33 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:33.594949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badce16-666d-4e43-afc1-c0656464cc5e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:33.595628 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-38f727a7-8751-4f44-9414-608a347e5cb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be95dc5aa8fb4c639e257b09f9ba38ef', 'request_id': 'req-38f727a7-8751-4f44-9414-608a347e5cb4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.600384 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:33.604591 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-38f727a7-8751-4f44-9414-608a347e5cb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be95dc5aa8fb4c639e257b09f9ba38ef', 'request_id': 'req-38f727a7-8751-4f44-9414-608a347e5cb4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.620587 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-38f727a7-8751-4f44-9414-608a347e5cb4 admin admin] The token cache is being invalidate because project be95dc5aa8fb4c639e257b09f9ba38ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:24:33.621202 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-38f727a7-8751-4f44-9414-608a347e5cb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.622188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-38f727a7-8751-4f44-9414-608a347e5cb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.623947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1091/2183] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:24:33 2022] DELETE /identity/v3/projects/be95dc5aa8fb4c639e257b09f9ba38ef => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:33.629178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b194ef-a978-4318-8825-22207eb3e683 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:33.629663 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b194ef-a978-4318-8825-22207eb3e683 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:33.630144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b194ef-a978-4318-8825-22207eb3e683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:33.654845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-97b194ef-a978-4318-8825-22207eb3e683 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:33.657118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-97b194ef-a978-4318-8825-22207eb3e683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:33.666902 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badce16-666d-4e43-afc1-c0656464cc5e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:33.667457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1092/2184] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:33.675172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ffd31-89a7-4f9a-949b-f996804d8424 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:33.675709 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ffd31-89a7-4f9a-949b-f996804d8424 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:33.676164 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ffd31-89a7-4f9a-949b-f996804d8424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:33.678236 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862369d-1695-49ab-983a-61e150f8914c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:33.684216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862369d-1695-49ab-983a-61e150f8914c admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:33.684512 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862369d-1695-49ab-983a-61e150f8914c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:33.684793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862369d-1695-49ab-983a-61e150f8914c admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:33.685449 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862369d-1695-49ab-983a-61e150f8914c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:33.687384 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862369d-1695-49ab-983a-61e150f8914c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:33.702005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-1d2ffd31-89a7-4f9a-949b-f996804d8424 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:33.703241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1d2ffd31-89a7-4f9a-949b-f996804d8424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:33.711050 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1093/2185] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:24:33 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:24:33.715663 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1093/2186] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:33.718454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ffd31-89a7-4f9a-949b-f996804d8424 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:33.718576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862369d-1695-49ab-983a-61e150f8914c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:33.722738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2862369d-1695-49ab-983a-61e150f8914c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:33.729889 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ffd31-89a7-4f9a-949b-f996804d8424 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:33.734809 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862369d-1695-49ab-983a-61e150f8914c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:33.739281 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75072fe-09cd-4c10-89db-ed787f7ebda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:33.740372 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75072fe-09cd-4c10-89db-ed787f7ebda1 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:33.740711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75072fe-09cd-4c10-89db-ed787f7ebda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:33.741083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75072fe-09cd-4c10-89db-ed787f7ebda1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:33.741783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75072fe-09cd-4c10-89db-ed787f7ebda1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:24:33.741936 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daeeab8-f033-4d3b-89ea-7b1a69fa7caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:33.742195 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75072fe-09cd-4c10-89db-ed787f7ebda1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:33.742456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daeeab8-f033-4d3b-89ea-7b1a69fa7caf admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:33.742745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daeeab8-f033-4d3b-89ea-7b1a69fa7caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:33.743098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daeeab8-f033-4d3b-89ea-7b1a69fa7caf admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:33.744081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daeeab8-f033-4d3b-89ea-7b1a69fa7caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:33.744222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75072fe-09cd-4c10-89db-ed787f7ebda1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:33.745685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daeeab8-f033-4d3b-89ea-7b1a69fa7caf admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:33.756667 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1094/2187] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:33.761884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daeeab8-f033-4d3b-89ea-7b1a69fa7caf admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:33.762669 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1094/2188] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:24:33.769349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75072fe-09cd-4c10-89db-ed787f7ebda1 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:33.773462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daeeab8-f033-4d3b-89ea-7b1a69fa7caf admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:33.778361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bb029-5979-4fac-afd5-579b71dd5521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:33.778971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bb029-5979-4fac-afd5-579b71dd5521 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:33.779250 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bb029-5979-4fac-afd5-579b71dd5521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:33.779543 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bb029-5979-4fac-afd5-579b71dd5521 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:33.779819 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75072fe-09cd-4c10-89db-ed787f7ebda1 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:33.780244 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bb029-5979-4fac-afd5-579b71dd5521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:33.782082 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bb029-5979-4fac-afd5-579b71dd5521 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:33.787287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86013110-3e65-4be7-a08f-7c161aef662b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:33.788179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86013110-3e65-4be7-a08f-7c161aef662b admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:33.788678 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86013110-3e65-4be7-a08f-7c161aef662b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:33.789135 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86013110-3e65-4be7-a08f-7c161aef662b admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:33.789964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86013110-3e65-4be7-a08f-7c161aef662b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:33.792937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86013110-3e65-4be7-a08f-7c161aef662b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:33.793242 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1095/2189] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:33.796491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bb029-5979-4fac-afd5-579b71dd5521 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:33.798185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-287bb029-5979-4fac-afd5-579b71dd5521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:33.811968 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bb029-5979-4fac-afd5-579b71dd5521 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:33.817156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e5234-8fb4-4e95-b787-f0f02d68c398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:33.817821 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e5234-8fb4-4e95-b787-f0f02d68c398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:33.818177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e5234-8fb4-4e95-b787-f0f02d68c398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:33.818521 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e5234-8fb4-4e95-b787-f0f02d68c398 admin admin] PATH_INFO: `/v3/projects/c1d6cd42c9874f7691bff03bb4375b5e` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:33.820354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e5234-8fb4-4e95-b787-f0f02d68c398 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1d6cd42c9874f7691bff03bb4375b5e)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:33.822266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e5234-8fb4-4e95-b787-f0f02d68c398 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:33.826414 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1095/2190] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:24:33 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:24:33.827048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86013110-3e65-4be7-a08f-7c161aef662b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:33.883949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d5e5234-8fb4-4e95-b787-f0f02d68c398 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1d6cd42c9874f7691bff03bb4375b5e', 'request_id': 'req-2d5e5234-8fb4-4e95-b787-f0f02d68c398'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.893581 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d5e5234-8fb4-4e95-b787-f0f02d68c398 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1d6cd42c9874f7691bff03bb4375b5e', 'request_id': 'req-2d5e5234-8fb4-4e95-b787-f0f02d68c398'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.901155 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d5e5234-8fb4-4e95-b787-f0f02d68c398 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1d6cd42c9874f7691bff03bb4375b5e', 'request_id': 'req-2d5e5234-8fb4-4e95-b787-f0f02d68c398'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.912637 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d5e5234-8fb4-4e95-b787-f0f02d68c398 admin admin] The token cache is being invalidate because project c1d6cd42c9874f7691bff03bb4375b5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:24:33.912943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d5e5234-8fb4-4e95-b787-f0f02d68c398 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.913654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2d5e5234-8fb4-4e95-b787-f0f02d68c398 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:33.915411 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1096/2191] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:24:33 2022] DELETE /identity/v3/projects/c1d6cd42c9874f7691bff03bb4375b5e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:33.918942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e5234-8fb4-4e95-b787-f0f02d68c398 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:33.922022 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2d5e5234-8fb4-4e95-b787-f0f02d68c398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:33.924724 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86013110-3e65-4be7-a08f-7c161aef662b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:33.980017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:33.990228 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce72a6-bd5c-4442-960e-9f94e00f035d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:33.991229 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce72a6-bd5c-4442-960e-9f94e00f035d admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:33.991609 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce72a6-bd5c-4442-960e-9f94e00f035d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:33.991893 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce72a6-bd5c-4442-960e-9f94e00f035d admin admin] PATH_INFO: `/v3/projects/53e1ba5fc96344cebdb16d6a3e50ab8f/users/8d817e129beb41168af70c3ada99aeda/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:33.998464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e5234-8fb4-4e95-b787-f0f02d68c398 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:34.004771 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84e9cb-ff96-4b4c-9edb-763331b16490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:34.005391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84e9cb-ff96-4b4c-9edb-763331b16490 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:34.005897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84e9cb-ff96-4b4c-9edb-763331b16490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:34.006188 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84e9cb-ff96-4b4c-9edb-763331b16490 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:34.006824 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b84e9cb-ff96-4b4c-9edb-763331b16490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:34.009364 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b84e9cb-ff96-4b4c-9edb-763331b16490 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:34.010113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce72a6-bd5c-4442-960e-9f94e00f035d admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e1ba5fc96344cebdb16d6a3e50ab8f, user_id=8d817e129beb41168af70c3ada99aeda, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:34.013209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce72a6-bd5c-4442-960e-9f94e00f035d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:34.028308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1096/2192] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:24:33 2022] PUT /identity/v3/projects/53e1ba5fc96344cebdb16d6a3e50ab8f/users/8d817e129beb41168af70c3ada99aeda/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:34.029771 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce72a6-bd5c-4442-960e-9f94e00f035d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:34.031704 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-56ce72a6-bd5c-4442-960e-9f94e00f035d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:34.035776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1097/2193] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:24:33 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:34.037160 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3145a16-3545-457e-83d5-d9043e334c35 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:34.037451 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3145a16-3545-457e-83d5-d9043e334c35 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:34.038133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3145a16-3545-457e-83d5-d9043e334c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:34.048187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:34.064022 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c3145a16-3545-457e-83d5-d9043e334c35 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:34.065268 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c3145a16-3545-457e-83d5-d9043e334c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:34.123945 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce72a6-bd5c-4442-960e-9f94e00f035d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:34.129040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac8502-2ef4-42c9-8432-78627eda33c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:34.129344 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1098/2194] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:34.130188 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3145a16-3545-457e-83d5-d9043e334c35 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:34.133316 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac8502-2ef4-42c9-8432-78627eda33c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:34.133316 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac8502-2ef4-42c9-8432-78627eda33c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:34.133488 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac8502-2ef4-42c9-8432-78627eda33c2 admin admin] PATH_INFO: `/v3/users/134bede3d73845ccbb676fe0bcdd00b7` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:34.135049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac8502-2ef4-42c9-8432-78627eda33c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=134bede3d73845ccbb676fe0bcdd00b7)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:34.138447 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac8502-2ef4-42c9-8432-78627eda33c2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:34.141721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3145a16-3545-457e-83d5-d9043e334c35 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:34.151110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971d56b-414f-4213-a298-3823a858ce66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:34.152042 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971d56b-414f-4213-a298-3823a858ce66 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:34.152326 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971d56b-414f-4213-a298-3823a858ce66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:34.152649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971d56b-414f-4213-a298-3823a858ce66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:34.153703 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971d56b-414f-4213-a298-3823a858ce66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:34.156313 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971d56b-414f-4213-a298-3823a858ce66 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:34.169105 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1099/2195] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:34.169902 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971d56b-414f-4213-a298-3823a858ce66 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:34.171864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1971d56b-414f-4213-a298-3823a858ce66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:34.194606 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-afac8502-2ef4-42c9-8432-78627eda33c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '134bede3d73845ccbb676fe0bcdd00b7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:34.201649 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-afac8502-2ef4-42c9-8432-78627eda33c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '134bede3d73845ccbb676fe0bcdd00b7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:34.202403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-afac8502-2ef4-42c9-8432-78627eda33c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '134bede3d73845ccbb676fe0bcdd00b7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:34.203523 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-afac8502-2ef4-42c9-8432-78627eda33c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '134bede3d73845ccbb676fe0bcdd00b7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:34.213616 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-afac8502-2ef4-42c9-8432-78627eda33c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '134bede3d73845ccbb676fe0bcdd00b7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:34.233920 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1097/2196] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:24:34 2022] DELETE /identity/v3/users/134bede3d73845ccbb676fe0bcdd00b7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:34.235127 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac8502-2ef4-42c9-8432-78627eda33c2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:34.237827 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-afac8502-2ef4-42c9-8432-78627eda33c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:34.238539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971d56b-414f-4213-a298-3823a858ce66 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:34.310851 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa16de9-cd38-40ca-aa29-18677b4a573f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:34.311541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa16de9-cd38-40ca-aa29-18677b4a573f service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:34.311947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa16de9-cd38-40ca-aa29-18677b4a573f service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:34.312281 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa16de9-cd38-40ca-aa29-18677b4a573f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:34.316477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7fa16de9-cd38-40ca-aa29-18677b4a573f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:34.356443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac8502-2ef4-42c9-8432-78627eda33c2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:34.361840 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b64bc-771b-48b0-80e5-cd58186cb77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:34.362644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b64bc-771b-48b0-80e5-cd58186cb77a admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:34.362929 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b64bc-771b-48b0-80e5-cd58186cb77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:34.363201 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b64bc-771b-48b0-80e5-cd58186cb77a admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:34.363922 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b64bc-771b-48b0-80e5-cd58186cb77a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:34.365738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b64bc-771b-48b0-80e5-cd58186cb77a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:34.377870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:34.386264 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1098/2197] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:34.390376 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b64bc-771b-48b0-80e5-cd58186cb77a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:34.411450 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b64bc-771b-48b0-80e5-cd58186cb77a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:34.414496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa16de9-cd38-40ca-aa29-18677b4a573f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:34.416804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa16de9-cd38-40ca-aa29-18677b4a573f service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:34.428675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a6874-0b45-431c-a14b-de3aea8ff993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:34.429932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a6874-0b45-431c-a14b-de3aea8ff993 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:34.430227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a6874-0b45-431c-a14b-de3aea8ff993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:34.430483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a6874-0b45-431c-a14b-de3aea8ff993 admin admin] PATH_INFO: `/v3/projects/ed9cbbd61aa64f05a09915b991c9389f/users/29e9a32f7f7249468212d2208bf227d5/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:34.440515 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1100/2198] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:34.455341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a6874-0b45-431c-a14b-de3aea8ff993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed9cbbd61aa64f05a09915b991c9389f, user_id=29e9a32f7f7249468212d2208bf227d5, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:34.463185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a6874-0b45-431c-a14b-de3aea8ff993 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:34.492539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:34.516326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1099/2199] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:24:34 2022] PUT /identity/v3/projects/ed9cbbd61aa64f05a09915b991c9389f/users/29e9a32f7f7249468212d2208bf227d5/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:34.530308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ef5bd-8822-40b6-a32a-242bdbf97db3 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:34.530777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ef5bd-8822-40b6-a32a-242bdbf97db3 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:34.531223 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ef5bd-8822-40b6-a32a-242bdbf97db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:34.578531 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d52ef5bd-8822-40b6-a32a-242bdbf97db3 None None] MFA Rules not processed for user `29e9a32f7f7249468212d2208bf227d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:34.579828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d52ef5bd-8822-40b6-a32a-242bdbf97db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:34.686853 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1101/2200] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:34.700303 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a6874-0b45-431c-a14b-de3aea8ff993 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:34.703114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-901a6874-0b45-431c-a14b-de3aea8ff993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:34.718020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ef5bd-8822-40b6-a32a-242bdbf97db3 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:34.719752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d52ef5bd-8822-40b6-a32a-242bdbf97db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:34.809854 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a6874-0b45-431c-a14b-de3aea8ff993 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:34.814397 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47059be9-1ccf-420d-8fc8-2404ccbbb0f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:34.815012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47059be9-1ccf-420d-8fc8-2404ccbbb0f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:34.815293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47059be9-1ccf-420d-8fc8-2404ccbbb0f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:34.815604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47059be9-1ccf-420d-8fc8-2404ccbbb0f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:34.826702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ef5bd-8822-40b6-a32a-242bdbf97db3 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:34.828138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47059be9-1ccf-420d-8fc8-2404ccbbb0f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:34.830265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47059be9-1ccf-420d-8fc8-2404ccbbb0f1 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:34.832714 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac42955-4bdc-4feb-8bd0-0f05282b4e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:34.833420 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac42955-4bdc-4feb-8bd0-0f05282b4e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:34.834818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac42955-4bdc-4feb-8bd0-0f05282b4e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:34.835735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac42955-4bdc-4feb-8bd0-0f05282b4e84 admin admin] PATH_INFO: `/v3/projects/31128bb6082342fc956fb5ccbbe02b03` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:34.837072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac42955-4bdc-4feb-8bd0-0f05282b4e84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31128bb6082342fc956fb5ccbbe02b03)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:34.839471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac42955-4bdc-4feb-8bd0-0f05282b4e84 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:34.846284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1100/2201] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:34.938470 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1ac42955-4bdc-4feb-8bd0-0f05282b4e84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31128bb6082342fc956fb5ccbbe02b03', 'request_id': 'req-1ac42955-4bdc-4feb-8bd0-0f05282b4e84'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:34.947319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1ac42955-4bdc-4feb-8bd0-0f05282b4e84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31128bb6082342fc956fb5ccbbe02b03', 'request_id': 'req-1ac42955-4bdc-4feb-8bd0-0f05282b4e84'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:34.956522 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1ac42955-4bdc-4feb-8bd0-0f05282b4e84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31128bb6082342fc956fb5ccbbe02b03', 'request_id': 'req-1ac42955-4bdc-4feb-8bd0-0f05282b4e84'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:34.967130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:34.974790 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1ac42955-4bdc-4feb-8bd0-0f05282b4e84 admin admin] The token cache is being invalidate because project 31128bb6082342fc956fb5ccbbe02b03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:24:34.976712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1ac42955-4bdc-4feb-8bd0-0f05282b4e84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:34.978257 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1ac42955-4bdc-4feb-8bd0-0f05282b4e84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:34.982230 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1102/2202] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:24:34 2022] DELETE /identity/v3/projects/31128bb6082342fc956fb5ccbbe02b03 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:34.986994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a542a-28af-42dc-bd7c-f28734ffbe06 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:34.987546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a542a-28af-42dc-bd7c-f28734ffbe06 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:34.988197 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a542a-28af-42dc-bd7c-f28734ffbe06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:35.047747 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-534a542a-28af-42dc-bd7c-f28734ffbe06 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:35.052251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-534a542a-28af-42dc-bd7c-f28734ffbe06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:35.199982 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1101/2203] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:24:35.208717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3de74a-c5e4-411d-bb06-2c1768989e0c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:35.209007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3de74a-c5e4-411d-bb06-2c1768989e0c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:35.209291 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3de74a-c5e4-411d-bb06-2c1768989e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:35.254371 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ed3de74a-c5e4-411d-bb06-2c1768989e0c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:35.255795 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ed3de74a-c5e4-411d-bb06-2c1768989e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:35.276726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1103/2204] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:35.280558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a542a-28af-42dc-bd7c-f28734ffbe06 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:35.306473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a542a-28af-42dc-bd7c-f28734ffbe06 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:35.312154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e70dcb-e4b1-4f1e-ad19-c0ee1a98fbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:35.313410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e70dcb-e4b1-4f1e-ad19-c0ee1a98fbfd admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:35.313410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e70dcb-e4b1-4f1e-ad19-c0ee1a98fbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:35.313410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e70dcb-e4b1-4f1e-ad19-c0ee1a98fbfd admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:35.314294 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e70dcb-e4b1-4f1e-ad19-c0ee1a98fbfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:24:35.314806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e70dcb-e4b1-4f1e-ad19-c0ee1a98fbfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:35.317506 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e70dcb-e4b1-4f1e-ad19-c0ee1a98fbfd admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:35.333133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1102/2205] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:24:35.336529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3de74a-c5e4-411d-bb06-2c1768989e0c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:35.347533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3de74a-c5e4-411d-bb06-2c1768989e0c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:35.355310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43126ad-39ea-4234-be5b-acbb00ee7820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:35.356686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43126ad-39ea-4234-be5b-acbb00ee7820 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:35.356686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43126ad-39ea-4234-be5b-acbb00ee7820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:35.362724 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43126ad-39ea-4234-be5b-acbb00ee7820 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:35.363634 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43126ad-39ea-4234-be5b-acbb00ee7820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:35.368362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43126ad-39ea-4234-be5b-acbb00ee7820 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:35.414849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1104/2206] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:24:35 2022] POST /identity/v3/projects => generated 387 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:35.418660 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e70dcb-e4b1-4f1e-ad19-c0ee1a98fbfd admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:35.430771 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e70dcb-e4b1-4f1e-ad19-c0ee1a98fbfd admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:35.517992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7cbe1-5864-478d-ab92-0efc558be67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:35.520374 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7cbe1-5864-478d-ab92-0efc558be67e admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:35.521525 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7cbe1-5864-478d-ab92-0efc558be67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:35.521991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7cbe1-5864-478d-ab92-0efc558be67e admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:35.523464 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7cbe1-5864-478d-ab92-0efc558be67e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:35.526847 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7cbe1-5864-478d-ab92-0efc558be67e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:35.571445 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1103/2207] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:24:35 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:35.575473 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43126ad-39ea-4234-be5b-acbb00ee7820 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:35.587214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43126ad-39ea-4234-be5b-acbb00ee7820 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:35.594398 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d98a0a-2618-4fba-9c88-9fa2ab5f23ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:35.599953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d98a0a-2618-4fba-9c88-9fa2ab5f23ff admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:35.600318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d98a0a-2618-4fba-9c88-9fa2ab5f23ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:35.600692 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d98a0a-2618-4fba-9c88-9fa2ab5f23ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:35.602064 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d98a0a-2618-4fba-9c88-9fa2ab5f23ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:35.603885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d98a0a-2618-4fba-9c88-9fa2ab5f23ff admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:35.606760 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1086c-1eaa-408a-895e-7879b2ace528 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:35.607129 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1086c-1eaa-408a-895e-7879b2ace528 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:35.607483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1086c-1eaa-408a-895e-7879b2ace528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:35.632528 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1105/2208] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:35.637985 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d98a0a-2618-4fba-9c88-9fa2ab5f23ff admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:35.646690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-eac1086c-1eaa-408a-895e-7879b2ace528 None None] MFA Rules not processed for user `8d817e129beb41168af70c3ada99aeda`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:35.649442 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-eac1086c-1eaa-408a-895e-7879b2ace528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:35.651652 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d98a0a-2618-4fba-9c88-9fa2ab5f23ff admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:35.655064 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:35.660336 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2cd58-1d97-4234-831f-819b77b677e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:35.662434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2cd58-1d97-4234-831f-819b77b677e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:35.663311 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2cd58-1d97-4234-831f-819b77b677e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:35.664979 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2cd58-1d97-4234-831f-819b77b677e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:35.666009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2cd58-1d97-4234-831f-819b77b677e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:35.667949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2cd58-1d97-4234-831f-819b77b677e8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:35.685236 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1106/2209] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:35.686146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2cd58-1d97-4234-831f-819b77b677e8 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:35.699316 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2cd58-1d97-4234-831f-819b77b677e8 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:35.704130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d50cf-4c20-4694-83e3-0c5d22d5c7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:35.704817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d50cf-4c20-4694-83e3-0c5d22d5c7c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:35.705254 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d50cf-4c20-4694-83e3-0c5d22d5c7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:35.705555 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d50cf-4c20-4694-83e3-0c5d22d5c7c5 admin admin] PATH_INFO: `/v3/projects/1ac7c1f560b44ef1a62cb9467e061954/users/adabeb551cc04480a41fde030c26d6c2/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:35.724854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d50cf-4c20-4694-83e3-0c5d22d5c7c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ac7c1f560b44ef1a62cb9467e061954, user_id=adabeb551cc04480a41fde030c26d6c2, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:35.727622 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d50cf-4c20-4694-83e3-0c5d22d5c7c5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:35.750909 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1104/2210] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:24:35.754879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1107/2211] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:24:35 2022] PUT /identity/v3/projects/1ac7c1f560b44ef1a62cb9467e061954/users/adabeb551cc04480a41fde030c26d6c2/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:35.764894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d50cf-4c20-4694-83e3-0c5d22d5c7c5 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:35.765657 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bb788-8c05-4643-9f2a-4939c702d384 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:35.765953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bb788-8c05-4643-9f2a-4939c702d384 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:35.766231 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bb788-8c05-4643-9f2a-4939c702d384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:35.767932 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d91d50cf-4c20-4694-83e3-0c5d22d5c7c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:35.814119 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-1a0bb788-8c05-4643-9f2a-4939c702d384 None None] MFA Rules not processed for user `adabeb551cc04480a41fde030c26d6c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:35.815734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1a0bb788-8c05-4643-9f2a-4939c702d384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:35.855103 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d50cf-4c20-4694-83e3-0c5d22d5c7c5 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:35.866679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87f60d-18ef-4cdb-8505-3dc9d98cc8af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:35.868433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87f60d-18ef-4cdb-8505-3dc9d98cc8af service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:35.870095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87f60d-18ef-4cdb-8505-3dc9d98cc8af service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:35.870707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87f60d-18ef-4cdb-8505-3dc9d98cc8af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:35.910215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1105/2212] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:35.911128 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bb788-8c05-4643-9f2a-4939c702d384 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:35.951391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87f60d-18ef-4cdb-8505-3dc9d98cc8af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:35.953190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87f60d-18ef-4cdb-8505-3dc9d98cc8af service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:35.971894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1108/2213] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:35.977859 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bb788-8c05-4643-9f2a-4939c702d384 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:35.982647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c4b59-e860-422f-9f33-58e645897456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:35.983267 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c4b59-e860-422f-9f33-58e645897456 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:35.983558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c4b59-e860-422f-9f33-58e645897456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:35.983832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c4b59-e860-422f-9f33-58e645897456 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:35.984461 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c4b59-e860-422f-9f33-58e645897456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:35.986353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c4b59-e860-422f-9f33-58e645897456 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:36.021675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1106/2214] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:24:35 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:36.024250 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87f60d-18ef-4cdb-8505-3dc9d98cc8af service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:36.039274 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87f60d-18ef-4cdb-8505-3dc9d98cc8af service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:36.081708 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:36.111428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50019fd9-238e-4ef7-be10-12f026003d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:36.112134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50019fd9-238e-4ef7-be10-12f026003d1f admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:36.112425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50019fd9-238e-4ef7-be10-12f026003d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:36.112683 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50019fd9-238e-4ef7-be10-12f026003d1f admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:36.113334 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50019fd9-238e-4ef7-be10-12f026003d1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:36.115390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50019fd9-238e-4ef7-be10-12f026003d1f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:36.141670 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1109/2215] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:24:36 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:36.143302 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c4b59-e860-422f-9f33-58e645897456 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:36.154637 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c4b59-e860-422f-9f33-58e645897456 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:36.162315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0e5de-895c-4ce9-855b-74fc4e87d781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:36.162964 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0e5de-895c-4ce9-855b-74fc4e87d781 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:36.163805 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0e5de-895c-4ce9-855b-74fc4e87d781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:36.164083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0e5de-895c-4ce9-855b-74fc4e87d781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:36.164738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0e5de-895c-4ce9-855b-74fc4e87d781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:36.166453 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0e5de-895c-4ce9-855b-74fc4e87d781 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:36.178694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1107/2216] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:36.182778 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50019fd9-238e-4ef7-be10-12f026003d1f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:36.196899 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50019fd9-238e-4ef7-be10-12f026003d1f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:36.202816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d8db7-f5e3-485d-9c0b-be3fa3f8af2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:36.203431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d8db7-f5e3-485d-9c0b-be3fa3f8af2a admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:36.203745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d8db7-f5e3-485d-9c0b-be3fa3f8af2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:36.204042 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d8db7-f5e3-485d-9c0b-be3fa3f8af2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:36.204681 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d8db7-f5e3-485d-9c0b-be3fa3f8af2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:36.206304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d8db7-f5e3-485d-9c0b-be3fa3f8af2a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:36.217708 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1110/2217] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:36.221974 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0e5de-895c-4ce9-855b-74fc4e87d781 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:36.237718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0e5de-895c-4ce9-855b-74fc4e87d781 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:36.249950 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec804742-134b-437e-a3c7-5766f3f92e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:36.250762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec804742-134b-437e-a3c7-5766f3f92e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:36.251050 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec804742-134b-437e-a3c7-5766f3f92e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:36.251386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec804742-134b-437e-a3c7-5766f3f92e35 admin admin] PATH_INFO: `/v3/projects/aeddf47fdd5146fea6246fa8260cef07/users/73317c86201744d8a4558467032c2b3e/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:36.284931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:36.301031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec804742-134b-437e-a3c7-5766f3f92e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeddf47fdd5146fea6246fa8260cef07, user_id=73317c86201744d8a4558467032c2b3e, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:36.305313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec804742-134b-437e-a3c7-5766f3f92e35 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:36.332161 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1108/2218] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:24:36 2022] PUT /identity/v3/projects/aeddf47fdd5146fea6246fa8260cef07/users/73317c86201744d8a4558467032c2b3e/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:36.334044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf57bf-8b56-4695-a5b9-b0f537ec2a53 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:36.334044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf57bf-8b56-4695-a5b9-b0f537ec2a53 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:36.334257 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf57bf-8b56-4695-a5b9-b0f537ec2a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:36.388879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-66cf57bf-8b56-4695-a5b9-b0f537ec2a53 None None] MFA Rules not processed for user `73317c86201744d8a4558467032c2b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:36.390178 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-66cf57bf-8b56-4695-a5b9-b0f537ec2a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:36.523573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1111/2219] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:36.529118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec804742-134b-437e-a3c7-5766f3f92e35 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:36.600427 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec804742-134b-437e-a3c7-5766f3f92e35 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:36.606141 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c72ad-6670-48c0-b4a8-10a787c846fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:36.606722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c72ad-6670-48c0-b4a8-10a787c846fb admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:36.607052 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c72ad-6670-48c0-b4a8-10a787c846fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:36.607361 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c72ad-6670-48c0-b4a8-10a787c846fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:36.608151 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c72ad-6670-48c0-b4a8-10a787c846fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:36.610060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c72ad-6670-48c0-b4a8-10a787c846fb admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:36.637144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1109/2220] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:24:36 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:24:36.640995 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf57bf-8b56-4695-a5b9-b0f537ec2a53 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:36.654940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf57bf-8b56-4695-a5b9-b0f537ec2a53 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:36.722883 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87274d1c-5a32-4dc7-ad13-6dbc1cae1e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:36.723646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87274d1c-5a32-4dc7-ad13-6dbc1cae1e20 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:36.724178 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87274d1c-5a32-4dc7-ad13-6dbc1cae1e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:36.724361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87274d1c-5a32-4dc7-ad13-6dbc1cae1e20 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:36.725078 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87274d1c-5a32-4dc7-ad13-6dbc1cae1e20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:36.727436 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87274d1c-5a32-4dc7-ad13-6dbc1cae1e20 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:36.755269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1112/2221] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:24:36 2022] POST /identity/v3/users => generated 300 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:36.760567 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c72ad-6670-48c0-b4a8-10a787c846fb admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:36.797980 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c72ad-6670-48c0-b4a8-10a787c846fb admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:36.809083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f7eed-fbe9-4e34-9da8-d4bbf8bb7dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:36.809923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f7eed-fbe9-4e34-9da8-d4bbf8bb7dff admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:36.810296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f7eed-fbe9-4e34-9da8-d4bbf8bb7dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:36.810650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f7eed-fbe9-4e34-9da8-d4bbf8bb7dff admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:36.811400 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f7eed-fbe9-4e34-9da8-d4bbf8bb7dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:36.814186 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f7eed-fbe9-4e34-9da8-d4bbf8bb7dff admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:36.831547 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1110/2222] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:36.836131 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87274d1c-5a32-4dc7-ad13-6dbc1cae1e20 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:36.849232 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87274d1c-5a32-4dc7-ad13-6dbc1cae1e20 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:36.856516 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b907d-aeb5-430c-b4d8-2e5673e3d8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:36.857199 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b907d-aeb5-430c-b4d8-2e5673e3d8f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:36.857483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b907d-aeb5-430c-b4d8-2e5673e3d8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:36.857768 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b907d-aeb5-430c-b4d8-2e5673e3d8f2 admin admin] PATH_INFO: `/v3/projects/89a00b05b69942529ff2b4b95affabd6/users/8646611b2d834d01a7bac6fa0330ec59/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:36.873619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b907d-aeb5-430c-b4d8-2e5673e3d8f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89a00b05b69942529ff2b4b95affabd6, user_id=8646611b2d834d01a7bac6fa0330ec59, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:36.876515 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b907d-aeb5-430c-b4d8-2e5673e3d8f2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:36.896401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1113/2223] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:24:36 2022] PUT /identity/v3/projects/89a00b05b69942529ff2b4b95affabd6/users/8646611b2d834d01a7bac6fa0330ec59/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:36.898292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f7eed-fbe9-4e34-9da8-d4bbf8bb7dff admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:36.910407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f7eed-fbe9-4e34-9da8-d4bbf8bb7dff admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:36.987416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666997d-3c48-4fa8-8bc7-fd38beaa6716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:36.988195 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666997d-3c48-4fa8-8bc7-fd38beaa6716 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:36.988526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666997d-3c48-4fa8-8bc7-fd38beaa6716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:36.989353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666997d-3c48-4fa8-8bc7-fd38beaa6716 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:36.992364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666997d-3c48-4fa8-8bc7-fd38beaa6716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:36.994626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666997d-3c48-4fa8-8bc7-fd38beaa6716 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:37.016356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1111/2224] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:37.020763 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b907d-aeb5-430c-b4d8-2e5673e3d8f2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:37.032722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b907d-aeb5-430c-b4d8-2e5673e3d8f2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:37.039317 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805c63b-f58a-4752-aae6-8d6319228904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:37.040154 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805c63b-f58a-4752-aae6-8d6319228904 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:37.040685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805c63b-f58a-4752-aae6-8d6319228904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:37.041057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805c63b-f58a-4752-aae6-8d6319228904 admin admin] PATH_INFO: `/v3/projects/89a00b05b69942529ff2b4b95affabd6/users/8646611b2d834d01a7bac6fa0330ec59/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:37.043565 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805c63b-f58a-4752-aae6-8d6319228904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89a00b05b69942529ff2b4b95affabd6, user_id=8646611b2d834d01a7bac6fa0330ec59, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:37.046891 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805c63b-f58a-4752-aae6-8d6319228904 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:37.053467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:37.062372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1114/2225] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:24:37 2022] PUT /identity/v3/projects/89a00b05b69942529ff2b4b95affabd6/users/8646611b2d834d01a7bac6fa0330ec59/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:37.071418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c921cf6-c9e6-4c2b-9fee-3438282a2778 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:37.071737 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c921cf6-c9e6-4c2b-9fee-3438282a2778 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:37.072024 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c921cf6-c9e6-4c2b-9fee-3438282a2778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:37.116968 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3c921cf6-c9e6-4c2b-9fee-3438282a2778 None None] MFA Rules not processed for user `8646611b2d834d01a7bac6fa0330ec59`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:37.118219 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3c921cf6-c9e6-4c2b-9fee-3438282a2778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:37.174744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:37.208621 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1112/2226] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:37.220675 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805c63b-f58a-4752-aae6-8d6319228904 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:37.309010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805c63b-f58a-4752-aae6-8d6319228904 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:37.313711 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089325d0-8706-4bd3-8057-22b1d4309c4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:37.314338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089325d0-8706-4bd3-8057-22b1d4309c4a service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:37.314785 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089325d0-8706-4bd3-8057-22b1d4309c4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:37.315059 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089325d0-8706-4bd3-8057-22b1d4309c4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:37.325673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089325d0-8706-4bd3-8057-22b1d4309c4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:37.327269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089325d0-8706-4bd3-8057-22b1d4309c4a service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:37.340153 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1115/2227] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:24:37.432428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a98728-5941-46fa-8370-9fc0752b0d31 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:37.432738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a98728-5941-46fa-8370-9fc0752b0d31 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:37.433020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a98728-5941-46fa-8370-9fc0752b0d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:37.461509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-76a98728-5941-46fa-8370-9fc0752b0d31 None None] MFA Rules not processed for user `8d817e129beb41168af70c3ada99aeda`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:37.472697 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-76a98728-5941-46fa-8370-9fc0752b0d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:37.590603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1113/2228] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:24:37.639680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089325d0-8706-4bd3-8057-22b1d4309c4a service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:37.644670 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-089325d0-8706-4bd3-8057-22b1d4309c4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:37.786565 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089325d0-8706-4bd3-8057-22b1d4309c4a service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:37.792757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91881e91-ec7c-4c85-91f1-eb03f165ba1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:37.793394 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91881e91-ec7c-4c85-91f1-eb03f165ba1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:37.793672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91881e91-ec7c-4c85-91f1-eb03f165ba1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:37.793940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91881e91-ec7c-4c85-91f1-eb03f165ba1f admin admin] PATH_INFO: `/v3/users/18d196263d2c49f88b5f81acb2586bc1` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:37.795353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91881e91-ec7c-4c85-91f1-eb03f165ba1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=18d196263d2c49f88b5f81acb2586bc1)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:37.797437 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91881e91-ec7c-4c85-91f1-eb03f165ba1f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:37.857940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-91881e91-ec7c-4c85-91f1-eb03f165ba1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18d196263d2c49f88b5f81acb2586bc1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:37.865743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-91881e91-ec7c-4c85-91f1-eb03f165ba1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18d196263d2c49f88b5f81acb2586bc1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:37.866878 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-91881e91-ec7c-4c85-91f1-eb03f165ba1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18d196263d2c49f88b5f81acb2586bc1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:37.868014 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-91881e91-ec7c-4c85-91f1-eb03f165ba1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18d196263d2c49f88b5f81acb2586bc1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:37.877320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-91881e91-ec7c-4c85-91f1-eb03f165ba1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18d196263d2c49f88b5f81acb2586bc1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:37.898794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1116/2229] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:24:37 2022] DELETE /identity/v3/users/18d196263d2c49f88b5f81acb2586bc1 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:38.096353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a98728-5941-46fa-8370-9fc0752b0d31 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:38.112664 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a98728-5941-46fa-8370-9fc0752b0d31 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:38.165284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91881e91-ec7c-4c85-91f1-eb03f165ba1f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:38.166792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-91881e91-ec7c-4c85-91f1-eb03f165ba1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:38.182381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fab437-ab56-4ab8-9173-fe0dab5639f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:38.183020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fab437-ab56-4ab8-9173-fe0dab5639f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:38.183767 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fab437-ab56-4ab8-9173-fe0dab5639f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:38.184047 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fab437-ab56-4ab8-9173-fe0dab5639f7 admin admin] PATH_INFO: `/v3/projects/69d2acb456bc4aaeae1ff0927195aba2` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:38.185316 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fab437-ab56-4ab8-9173-fe0dab5639f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69d2acb456bc4aaeae1ff0927195aba2)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:38.187757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fab437-ab56-4ab8-9173-fe0dab5639f7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:38.191191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:38.235227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91881e91-ec7c-4c85-91f1-eb03f165ba1f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:38.245096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a2fab437-ab56-4ab8-9173-fe0dab5639f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69d2acb456bc4aaeae1ff0927195aba2', 'request_id': 'req-a2fab437-ab56-4ab8-9173-fe0dab5639f7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:38.258869 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a2fab437-ab56-4ab8-9173-fe0dab5639f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69d2acb456bc4aaeae1ff0927195aba2', 'request_id': 'req-a2fab437-ab56-4ab8-9173-fe0dab5639f7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:38.266647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a2fab437-ab56-4ab8-9173-fe0dab5639f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69d2acb456bc4aaeae1ff0927195aba2', 'request_id': 'req-a2fab437-ab56-4ab8-9173-fe0dab5639f7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:38.279405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a2fab437-ab56-4ab8-9173-fe0dab5639f7 admin admin] The token cache is being invalidate because project 69d2acb456bc4aaeae1ff0927195aba2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:24:38.279723 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a2fab437-ab56-4ab8-9173-fe0dab5639f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:38.280403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a2fab437-ab56-4ab8-9173-fe0dab5639f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:38.285758 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1114/2230] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:24:38 2022] DELETE /identity/v3/projects/69d2acb456bc4aaeae1ff0927195aba2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:38.287073 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aab270-080f-48bb-94b2-54542b4b2f1c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:38.287367 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aab270-080f-48bb-94b2-54542b4b2f1c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:38.288044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aab270-080f-48bb-94b2-54542b4b2f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:38.295438 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b12ea-f02f-4a9e-9e0b-4b3e80d2b783 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:38.296296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b12ea-f02f-4a9e-9e0b-4b3e80d2b783 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:38.296561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b12ea-f02f-4a9e-9e0b-4b3e80d2b783 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:38.296896 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b12ea-f02f-4a9e-9e0b-4b3e80d2b783 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:38.299019 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-801b12ea-f02f-4a9e-9e0b-4b3e80d2b783 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:38.308841 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-92aab270-080f-48bb-94b2-54542b4b2f1c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:38.310059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-92aab270-080f-48bb-94b2-54542b4b2f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:38.323020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:38.372690 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b12ea-f02f-4a9e-9e0b-4b3e80d2b783 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:38.374412 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b12ea-f02f-4a9e-9e0b-4b3e80d2b783 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:38.386845 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1117/2231] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:24:38.402661 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1115/2232] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:24:38.410577 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aad0e1-55fb-46ee-b57c-ab26efed6b80 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:38.410862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aad0e1-55fb-46ee-b57c-ab26efed6b80 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:38.411160 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aad0e1-55fb-46ee-b57c-ab26efed6b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:38.431686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-81aad0e1-55fb-46ee-b57c-ab26efed6b80 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:38.432988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-81aad0e1-55fb-46ee-b57c-ab26efed6b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:38.441542 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1118/2233] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:38.446491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aab270-080f-48bb-94b2-54542b4b2f1c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:38.458525 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aab270-080f-48bb-94b2-54542b4b2f1c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:38.464115 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08a3ab-c6bc-43c6-a21d-70077ca36443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:38.465043 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08a3ab-c6bc-43c6-a21d-70077ca36443 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:38.465460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08a3ab-c6bc-43c6-a21d-70077ca36443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:38.465805 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08a3ab-c6bc-43c6-a21d-70077ca36443 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:38.466565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08a3ab-c6bc-43c6-a21d-70077ca36443 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:24:38.466934 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08a3ab-c6bc-43c6-a21d-70077ca36443 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:38.468691 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08a3ab-c6bc-43c6-a21d-70077ca36443 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:38.490624 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aad0e1-55fb-46ee-b57c-ab26efed6b80 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:38.501322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aad0e1-55fb-46ee-b57c-ab26efed6b80 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:38.501561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1116/2234] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:24:38.507176 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4245a-a13f-48e2-bcd0-ae3cfb8617f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:38.508604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4245a-a13f-48e2-bcd0-ae3cfb8617f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:38.508894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4245a-a13f-48e2-bcd0-ae3cfb8617f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:38.509173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4245a-a13f-48e2-bcd0-ae3cfb8617f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:38.509981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4245a-a13f-48e2-bcd0-ae3cfb8617f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:38.512511 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4245a-a13f-48e2-bcd0-ae3cfb8617f4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:38.547707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1119/2235] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:24:38 2022] POST /identity/v3/projects => generated 389 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:24:38.552098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08a3ab-c6bc-43c6-a21d-70077ca36443 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:38.565105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08a3ab-c6bc-43c6-a21d-70077ca36443 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:38.648587 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78611f20-b419-4ed6-93f4-79344d93c379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:38.649805 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78611f20-b419-4ed6-93f4-79344d93c379 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:38.650073 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78611f20-b419-4ed6-93f4-79344d93c379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:38.650342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78611f20-b419-4ed6-93f4-79344d93c379 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:38.650971 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78611f20-b419-4ed6-93f4-79344d93c379 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:38.652962 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78611f20-b419-4ed6-93f4-79344d93c379 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:38.676211 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1117/2236] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:24:38 2022] POST /identity/v3/users => generated 295 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:38.681039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4245a-a13f-48e2-bcd0-ae3cfb8617f4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:38.693387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4245a-a13f-48e2-bcd0-ae3cfb8617f4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:38.699009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f39866-8819-4938-bc8d-67564c466cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:38.699656 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f39866-8819-4938-bc8d-67564c466cba admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:38.700060 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f39866-8819-4938-bc8d-67564c466cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:38.700197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f39866-8819-4938-bc8d-67564c466cba admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:38.700852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f39866-8819-4938-bc8d-67564c466cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:38.702588 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f39866-8819-4938-bc8d-67564c466cba admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:38.721077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1120/2237] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:38.721191 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78611f20-b419-4ed6-93f4-79344d93c379 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:38.732556 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78611f20-b419-4ed6-93f4-79344d93c379 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:38.739611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d1c7a-6985-4581-b96f-c56669c4c113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:38.740347 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d1c7a-6985-4581-b96f-c56669c4c113 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:38.740868 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d1c7a-6985-4581-b96f-c56669c4c113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:38.741896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d1c7a-6985-4581-b96f-c56669c4c113 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:38.742647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d1c7a-6985-4581-b96f-c56669c4c113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:38.744659 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d1c7a-6985-4581-b96f-c56669c4c113 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:38.768848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1118/2238] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:38.769540 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f39866-8819-4938-bc8d-67564c466cba admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:38.802202 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f39866-8819-4938-bc8d-67564c466cba admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:38.816261 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f7751-0c40-49bc-a2ed-7233512c3eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:38.817665 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f7751-0c40-49bc-a2ed-7233512c3eeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:38.817959 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f7751-0c40-49bc-a2ed-7233512c3eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:38.818281 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f7751-0c40-49bc-a2ed-7233512c3eeb admin admin] PATH_INFO: `/v3/projects/eda6037b1d4b4d31b7a7ad298452cfbb/users/ff9c115c7083441c8b363e9b685ef110/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:38.844014 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f7751-0c40-49bc-a2ed-7233512c3eeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda6037b1d4b4d31b7a7ad298452cfbb, user_id=ff9c115c7083441c8b363e9b685ef110, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:38.847697 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f7751-0c40-49bc-a2ed-7233512c3eeb admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:38.874111 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6d758-9734-408b-8b4f-fc3c88a5d091 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:38.874385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6d758-9734-408b-8b4f-fc3c88a5d091 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:38.875791 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6d758-9734-408b-8b4f-fc3c88a5d091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:38.880650 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1121/2239] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:24:38 2022] PUT /identity/v3/projects/eda6037b1d4b4d31b7a7ad298452cfbb/users/ff9c115c7083441c8b363e9b685ef110/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:24:38.882082 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f7751-0c40-49bc-a2ed-7233512c3eeb admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:38.889141 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ef3f7751-0c40-49bc-a2ed-7233512c3eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:38.902578 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6ae6d758-9734-408b-8b4f-fc3c88a5d091 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:38.904440 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6ae6d758-9734-408b-8b4f-fc3c88a5d091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:38.938531 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:38.996144 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f7751-0c40-49bc-a2ed-7233512c3eeb admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:39.003002 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11baffc-ddef-4f57-b539-6420c5ccf2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:39.003830 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11baffc-ddef-4f57-b539-6420c5ccf2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:39.004336 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11baffc-ddef-4f57-b539-6420c5ccf2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:39.004621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11baffc-ddef-4f57-b539-6420c5ccf2bd admin admin] PATH_INFO: `/v3/users/29e9a32f7f7249468212d2208bf227d5` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:39.006238 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11baffc-ddef-4f57-b539-6420c5ccf2bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=29e9a32f7f7249468212d2208bf227d5)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:39.008949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11baffc-ddef-4f57-b539-6420c5ccf2bd admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:39.033965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1119/2240] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:39.047418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6d758-9734-408b-8b4f-fc3c88a5d091 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:39.049953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6ae6d758-9734-408b-8b4f-fc3c88a5d091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:39.077770 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e11baffc-ddef-4f57-b539-6420c5ccf2bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29e9a32f7f7249468212d2208bf227d5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:39.083340 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:39.087762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e11baffc-ddef-4f57-b539-6420c5ccf2bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29e9a32f7f7249468212d2208bf227d5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:39.088641 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e11baffc-ddef-4f57-b539-6420c5ccf2bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29e9a32f7f7249468212d2208bf227d5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:39.089604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e11baffc-ddef-4f57-b539-6420c5ccf2bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29e9a32f7f7249468212d2208bf227d5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:39.104701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e11baffc-ddef-4f57-b539-6420c5ccf2bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29e9a32f7f7249468212d2208bf227d5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:39.122605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6d758-9734-408b-8b4f-fc3c88a5d091 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:39.124699 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1122/2241] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:24:38 2022] DELETE /identity/v3/users/29e9a32f7f7249468212d2208bf227d5 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:39.129416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053bfd0-fe6b-44eb-a287-9031ba5613c8 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:39.129780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053bfd0-fe6b-44eb-a287-9031ba5613c8 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:39.130238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b0402-6089-4d41-9f59-8ed7d8a05716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:39.130601 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053bfd0-fe6b-44eb-a287-9031ba5613c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:39.130857 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b0402-6089-4d41-9f59-8ed7d8a05716 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:39.131136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b0402-6089-4d41-9f59-8ed7d8a05716 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:39.131406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b0402-6089-4d41-9f59-8ed7d8a05716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:39.133846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-773b0402-6089-4d41-9f59-8ed7d8a05716 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:39.147068 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b0402-6089-4d41-9f59-8ed7d8a05716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:39.149486 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b0402-6089-4d41-9f59-8ed7d8a05716 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:39.155756 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-4053bfd0-fe6b-44eb-a287-9031ba5613c8 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:39.157018 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4053bfd0-fe6b-44eb-a287-9031ba5613c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:39.165331 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1120/2242] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:39.272909 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1123/2243] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:39.285627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b0402-6089-4d41-9f59-8ed7d8a05716 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:39.287215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-773b0402-6089-4d41-9f59-8ed7d8a05716 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:39.305107 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b0402-6089-4d41-9f59-8ed7d8a05716 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:39.379898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e5e66-f32f-42f5-b766-c9886b426d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:39.380517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e5e66-f32f-42f5-b766-c9886b426d2f service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:39.380804 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e5e66-f32f-42f5-b766-c9886b426d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:39.381077 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e5e66-f32f-42f5-b766-c9886b426d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:39.395788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e5e66-f32f-42f5-b766-c9886b426d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:39.398392 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e5e66-f32f-42f5-b766-c9886b426d2f service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:39.410184 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1121/2244] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:39.753567 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053bfd0-fe6b-44eb-a287-9031ba5613c8 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:39.755760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4053bfd0-fe6b-44eb-a287-9031ba5613c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:39.770586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053bfd0-fe6b-44eb-a287-9031ba5613c8 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:39.777206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e7363-7e99-4273-9557-25f762e78b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:39.778300 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e7363-7e99-4273-9557-25f762e78b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:39.778300 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e7363-7e99-4273-9557-25f762e78b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:39.778574 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e7363-7e99-4273-9557-25f762e78b16 admin admin] PATH_INFO: `/v3/projects/ed9cbbd61aa64f05a09915b991c9389f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:39.780061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e7363-7e99-4273-9557-25f762e78b16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed9cbbd61aa64f05a09915b991c9389f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:39.782844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e7363-7e99-4273-9557-25f762e78b16 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:39.832614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:39.842095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-be0e7363-7e99-4273-9557-25f762e78b16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed9cbbd61aa64f05a09915b991c9389f', 'request_id': 'req-be0e7363-7e99-4273-9557-25f762e78b16'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:39.856234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-be0e7363-7e99-4273-9557-25f762e78b16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed9cbbd61aa64f05a09915b991c9389f', 'request_id': 'req-be0e7363-7e99-4273-9557-25f762e78b16'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:39.868000 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-be0e7363-7e99-4273-9557-25f762e78b16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed9cbbd61aa64f05a09915b991c9389f', 'request_id': 'req-be0e7363-7e99-4273-9557-25f762e78b16'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:39.881150 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-be0e7363-7e99-4273-9557-25f762e78b16 admin admin] The token cache is being invalidate because project ed9cbbd61aa64f05a09915b991c9389f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:24:39.881768 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-be0e7363-7e99-4273-9557-25f762e78b16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:39.882431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-be0e7363-7e99-4273-9557-25f762e78b16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:39.888322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1124/2245] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:24:39 2022] DELETE /identity/v3/projects/ed9cbbd61aa64f05a09915b991c9389f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:39.889925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637936cc-faeb-4b3d-bf5b-62d0d0b98a21 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:39.890385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637936cc-faeb-4b3d-bf5b-62d0d0b98a21 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:39.890465 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637936cc-faeb-4b3d-bf5b-62d0d0b98a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:39.945268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-637936cc-faeb-4b3d-bf5b-62d0d0b98a21 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:39.946530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-637936cc-faeb-4b3d-bf5b-62d0d0b98a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:40.079026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1122/2246] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:40.087763 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbee983-c94e-4b4d-ad89-21fd8d759722 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:40.088095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbee983-c94e-4b4d-ad89-21fd8d759722 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:40.088451 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbee983-c94e-4b4d-ad89-21fd8d759722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:40.125549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7fbee983-c94e-4b4d-ad89-21fd8d759722 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:40.129291 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7fbee983-c94e-4b4d-ad89-21fd8d759722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:40.156433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1125/2247] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:40.160778 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637936cc-faeb-4b3d-bf5b-62d0d0b98a21 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:40.172475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637936cc-faeb-4b3d-bf5b-62d0d0b98a21 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:40.177913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d28dd-e7ed-4243-91ed-22d88cdaab3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:40.178874 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d28dd-e7ed-4243-91ed-22d88cdaab3e admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:40.179227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d28dd-e7ed-4243-91ed-22d88cdaab3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:40.179641 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d28dd-e7ed-4243-91ed-22d88cdaab3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:40.180435 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d28dd-e7ed-4243-91ed-22d88cdaab3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:24:40.181303 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d28dd-e7ed-4243-91ed-22d88cdaab3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:40.183002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d28dd-e7ed-4243-91ed-22d88cdaab3e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:40.203181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbee983-c94e-4b4d-ad89-21fd8d759722 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:40.214959 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1123/2248] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:24:40.216216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbee983-c94e-4b4d-ad89-21fd8d759722 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:40.222077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4e28f-ebce-4782-9fe2-6e102c892fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:40.222961 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4e28f-ebce-4782-9fe2-6e102c892fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:40.223580 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4e28f-ebce-4782-9fe2-6e102c892fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:40.223955 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4e28f-ebce-4782-9fe2-6e102c892fb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:40.224893 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4e28f-ebce-4782-9fe2-6e102c892fb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:40.227968 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4e28f-ebce-4782-9fe2-6e102c892fb9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:40.274246 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d28dd-e7ed-4243-91ed-22d88cdaab3e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:40.275418 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1126/2249] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:24:40 2022] POST /identity/v3/projects => generated 369 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:40.289958 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d28dd-e7ed-4243-91ed-22d88cdaab3e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:40.358245 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:40.367974 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856708f3-a123-48cc-ac7b-6789ce6a47d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:40.368752 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856708f3-a123-48cc-ac7b-6789ce6a47d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:40.369879 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856708f3-a123-48cc-ac7b-6789ce6a47d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:40.370104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856708f3-a123-48cc-ac7b-6789ce6a47d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:40.370825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856708f3-a123-48cc-ac7b-6789ce6a47d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:40.373851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856708f3-a123-48cc-ac7b-6789ce6a47d5 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:40.403544 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1124/2250] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:24:40 2022] POST /identity/v3/users => generated 285 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:40.405789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4e28f-ebce-4782-9fe2-6e102c892fb9 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:40.417676 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4e28f-ebce-4782-9fe2-6e102c892fb9 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:40.422736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8836e-e37f-4971-b297-699401d6558c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:40.423456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8836e-e37f-4971-b297-699401d6558c admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:40.423995 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8836e-e37f-4971-b297-699401d6558c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:40.424401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8836e-e37f-4971-b297-699401d6558c admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:40.425272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8836e-e37f-4971-b297-699401d6558c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:40.427074 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8836e-e37f-4971-b297-699401d6558c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:40.439302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1127/2251] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:40.444065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856708f3-a123-48cc-ac7b-6789ce6a47d5 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:40.457637 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856708f3-a123-48cc-ac7b-6789ce6a47d5 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:40.462969 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c112781-f8d4-4262-bb96-bc380a700bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:40.463906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c112781-f8d4-4262-bb96-bc380a700bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:40.464051 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c112781-f8d4-4262-bb96-bc380a700bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:40.464971 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c112781-f8d4-4262-bb96-bc380a700bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:40.464971 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c112781-f8d4-4262-bb96-bc380a700bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:40.466997 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c112781-f8d4-4262-bb96-bc380a700bd8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:40.482629 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1125/2252] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:40.483754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8836e-e37f-4971-b297-699401d6558c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:40.497692 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8836e-e37f-4971-b297-699401d6558c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:40.503027 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f30e3-ea74-4897-b54f-1338ca5104f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:40.503739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f30e3-ea74-4897-b54f-1338ca5104f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:40.504005 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f30e3-ea74-4897-b54f-1338ca5104f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:40.504281 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f30e3-ea74-4897-b54f-1338ca5104f2 admin admin] PATH_INFO: `/v3/projects/6067d5ae59724b17858fd5ab3a145bf5/users/9aa8cfac6e1e43d9aed573bd77893e1e/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:40.520302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f30e3-ea74-4897-b54f-1338ca5104f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6067d5ae59724b17858fd5ab3a145bf5, user_id=9aa8cfac6e1e43d9aed573bd77893e1e, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:40.523282 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f30e3-ea74-4897-b54f-1338ca5104f2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:40.543212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1128/2253] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:24:40 2022] PUT /identity/v3/projects/6067d5ae59724b17858fd5ab3a145bf5/users/9aa8cfac6e1e43d9aed573bd77893e1e/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:24:40.554362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54ce91-dafb-495f-8ddd-ec27b1fcc50e None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:40.554648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54ce91-dafb-495f-8ddd-ec27b1fcc50e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:40.554919 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54ce91-dafb-495f-8ddd-ec27b1fcc50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:40.585828 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3e54ce91-dafb-495f-8ddd-ec27b1fcc50e None None] MFA Rules not processed for user `9aa8cfac6e1e43d9aed573bd77893e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:40.587120 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3e54ce91-dafb-495f-8ddd-ec27b1fcc50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:40.669271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1126/2254] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:40.688859 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f30e3-ea74-4897-b54f-1338ca5104f2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:40.690888 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-970f30e3-ea74-4897-b54f-1338ca5104f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:40.755588 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f30e3-ea74-4897-b54f-1338ca5104f2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:40.760038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962a330-3f46-42a3-a1b5-f9fa35dc38f1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a889616f86346f48b7573969174ee57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:40.760732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962a330-3f46-42a3-a1b5-f9fa35dc38f1 service glance] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:40.761066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962a330-3f46-42a3-a1b5-f9fa35dc38f1 service glance] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:40.761405 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962a330-3f46-42a3-a1b5-f9fa35dc38f1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:40.770792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962a330-3f46-42a3-a1b5-f9fa35dc38f1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:40.772393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962a330-3f46-42a3-a1b5-f9fa35dc38f1 service glance] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:40.783029 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1129/2255] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:43.932193 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ead93c-0fa5-4676-a0e1-e5733f2ee7bf None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:43.933201 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ead93c-0fa5-4676-a0e1-e5733f2ee7bf None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:43.933828 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ead93c-0fa5-4676-a0e1-e5733f2ee7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:43.972732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-67ead93c-0fa5-4676-a0e1-e5733f2ee7bf None None] MFA Rules not processed for user `ff9c115c7083441c8b363e9b685ef110`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:43.974050 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-67ead93c-0fa5-4676-a0e1-e5733f2ee7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:44.139394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1127/2256] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:44.219591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962a330-3f46-42a3-a1b5-f9fa35dc38f1 service glance] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:44.221783 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d962a330-3f46-42a3-a1b5-f9fa35dc38f1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:44.330709 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962a330-3f46-42a3-a1b5-f9fa35dc38f1 service glance] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:44.339011 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f27ef8-9f8d-427a-beb3-d0bdd5a33a18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:44.341226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f27ef8-9f8d-427a-beb3-d0bdd5a33a18 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:44.341226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f27ef8-9f8d-427a-beb3-d0bdd5a33a18 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:44.341568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f27ef8-9f8d-427a-beb3-d0bdd5a33a18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:44.348116 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:44.352343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f27ef8-9f8d-427a-beb3-d0bdd5a33a18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:44.354645 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f27ef8-9f8d-427a-beb3-d0bdd5a33a18 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:44.368712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1130/2257] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:46.269506 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca0ecd-4f11-4a9c-8840-9e2f685f36bc None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:46.270034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca0ecd-4f11-4a9c-8840-9e2f685f36bc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:46.270034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca0ecd-4f11-4a9c-8840-9e2f685f36bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:46.293730 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3eca0ecd-4f11-4a9c-8840-9e2f685f36bc None None] MFA Rules not processed for user `ff9c115c7083441c8b363e9b685ef110`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:46.294973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3eca0ecd-4f11-4a9c-8840-9e2f685f36bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:46.302486 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1128/2258] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:50.637987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f27ef8-9f8d-427a-beb3-d0bdd5a33a18 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:50.640119 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-72f27ef8-9f8d-427a-beb3-d0bdd5a33a18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:50.710657 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f27ef8-9f8d-427a-beb3-d0bdd5a33a18 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:50.714498 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd83ebe-2dd7-4f03-a295-b88eed95bef4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:50.715135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd83ebe-2dd7-4f03-a295-b88eed95bef4 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:50.715658 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd83ebe-2dd7-4f03-a295-b88eed95bef4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:50.715939 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd83ebe-2dd7-4f03-a295-b88eed95bef4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:50.717666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1dd83ebe-2dd7-4f03-a295-b88eed95bef4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:50.780943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd83ebe-2dd7-4f03-a295-b88eed95bef4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:50.782651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd83ebe-2dd7-4f03-a295-b88eed95bef4 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:50.792988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1131/2259] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:52.972136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca0ecd-4f11-4a9c-8840-9e2f685f36bc None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:52.973580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3eca0ecd-4f11-4a9c-8840-9e2f685f36bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:53.022124 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:53.038599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca0ecd-4f11-4a9c-8840-9e2f685f36bc None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:53.043428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f119e62-9c10-4c06-9a76-f3371a0b757f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:53.044117 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f119e62-9c10-4c06-9a76-f3371a0b757f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:53.044519 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f119e62-9c10-4c06-9a76-f3371a0b757f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:53.044801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f119e62-9c10-4c06-9a76-f3371a0b757f admin admin] PATH_INFO: `/v3/users/adabeb551cc04480a41fde030c26d6c2` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:53.046104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f119e62-9c10-4c06-9a76-f3371a0b757f admin admin] RBAC: Authorizing `identity:delete_user(user_id=adabeb551cc04480a41fde030c26d6c2)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:53.048143 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f119e62-9c10-4c06-9a76-f3371a0b757f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:53.095288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2f119e62-9c10-4c06-9a76-f3371a0b757f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adabeb551cc04480a41fde030c26d6c2'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:53.102955 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2f119e62-9c10-4c06-9a76-f3371a0b757f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adabeb551cc04480a41fde030c26d6c2'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:53.103959 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2f119e62-9c10-4c06-9a76-f3371a0b757f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adabeb551cc04480a41fde030c26d6c2'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:53.104625 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2f119e62-9c10-4c06-9a76-f3371a0b757f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adabeb551cc04480a41fde030c26d6c2'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:53.112897 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2f119e62-9c10-4c06-9a76-f3371a0b757f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adabeb551cc04480a41fde030c26d6c2'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:53.133245 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1129/2260] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:24:52 2022] DELETE /identity/v3/users/adabeb551cc04480a41fde030c26d6c2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:53.142756 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50526596-5988-4c7a-a4a2-3ff926b9d26d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:53.143304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50526596-5988-4c7a-a4a2-3ff926b9d26d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:53.143485 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50526596-5988-4c7a-a4a2-3ff926b9d26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:53.163734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-50526596-5988-4c7a-a4a2-3ff926b9d26d None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:53.164993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-50526596-5988-4c7a-a4a2-3ff926b9d26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:53.225921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1132/2261] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:53.235531 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f119e62-9c10-4c06-9a76-f3371a0b757f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:53.236976 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2f119e62-9c10-4c06-9a76-f3371a0b757f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:53.301635 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f119e62-9c10-4c06-9a76-f3371a0b757f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:53.305980 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c4d50-95ec-4c08-86c3-bc22463e9b93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:53.307318 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c4d50-95ec-4c08-86c3-bc22463e9b93 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:53.307708 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c4d50-95ec-4c08-86c3-bc22463e9b93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:53.308001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c4d50-95ec-4c08-86c3-bc22463e9b93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:53.318118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c4d50-95ec-4c08-86c3-bc22463e9b93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:53.320173 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c4d50-95ec-4c08-86c3-bc22463e9b93 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:53.329959 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1130/2262] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:53.636214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50526596-5988-4c7a-a4a2-3ff926b9d26d None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:53.638163 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-50526596-5988-4c7a-a4a2-3ff926b9d26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:53.661325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50526596-5988-4c7a-a4a2-3ff926b9d26d None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:53.667736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52419e-a4d5-47de-8297-3a4642618f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:53.668393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52419e-a4d5-47de-8297-3a4642618f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:53.668677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52419e-a4d5-47de-8297-3a4642618f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:53.668949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52419e-a4d5-47de-8297-3a4642618f90 admin admin] PATH_INFO: `/v3/projects/1ac7c1f560b44ef1a62cb9467e061954` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:53.670222 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52419e-a4d5-47de-8297-3a4642618f90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ac7c1f560b44ef1a62cb9467e061954)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:53.672221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52419e-a4d5-47de-8297-3a4642618f90 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:53.734092 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:53.738959 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c52419e-a4d5-47de-8297-3a4642618f90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ac7c1f560b44ef1a62cb9467e061954', 'request_id': 'req-8c52419e-a4d5-47de-8297-3a4642618f90'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:53.747752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c52419e-a4d5-47de-8297-3a4642618f90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ac7c1f560b44ef1a62cb9467e061954', 'request_id': 'req-8c52419e-a4d5-47de-8297-3a4642618f90'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:53.754582 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c52419e-a4d5-47de-8297-3a4642618f90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ac7c1f560b44ef1a62cb9467e061954', 'request_id': 'req-8c52419e-a4d5-47de-8297-3a4642618f90'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:53.766096 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c52419e-a4d5-47de-8297-3a4642618f90 admin admin] The token cache is being invalidate because project 1ac7c1f560b44ef1a62cb9467e061954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:24:53.766388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c52419e-a4d5-47de-8297-3a4642618f90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:53.767106 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c52419e-a4d5-47de-8297-3a4642618f90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:53.768623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1133/2263] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:24:53 2022] DELETE /identity/v3/projects/1ac7c1f560b44ef1a62cb9467e061954 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:53.772707 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c4d50-95ec-4c08-86c3-bc22463e9b93 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:53.783458 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c4d50-95ec-4c08-86c3-bc22463e9b93 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:53.844412 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf82e8-4b90-43bc-96c8-569eec598f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:53.845152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf82e8-4b90-43bc-96c8-569eec598f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:53.845445 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf82e8-4b90-43bc-96c8-569eec598f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:53.845724 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf82e8-4b90-43bc-96c8-569eec598f43 admin admin] PATH_INFO: `/v3/users/73317c86201744d8a4558467032c2b3e` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:53.846954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf82e8-4b90-43bc-96c8-569eec598f43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73317c86201744d8a4558467032c2b3e)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:53.848793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf82e8-4b90-43bc-96c8-569eec598f43 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:53.862261 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:53.899693 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-03cf82e8-4b90-43bc-96c8-569eec598f43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73317c86201744d8a4558467032c2b3e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:53.906389 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-03cf82e8-4b90-43bc-96c8-569eec598f43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73317c86201744d8a4558467032c2b3e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:53.907309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-03cf82e8-4b90-43bc-96c8-569eec598f43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73317c86201744d8a4558467032c2b3e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:53.907966 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-03cf82e8-4b90-43bc-96c8-569eec598f43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73317c86201744d8a4558467032c2b3e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:53.915742 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-03cf82e8-4b90-43bc-96c8-569eec598f43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73317c86201744d8a4558467032c2b3e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:53.933349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1131/2264] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:24:53 2022] DELETE /identity/v3/users/73317c86201744d8a4558467032c2b3e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:54.201056 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52419e-a4d5-47de-8297-3a4642618f90 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:54.202640 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8c52419e-a4d5-47de-8297-3a4642618f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:54.216618 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52419e-a4d5-47de-8297-3a4642618f90 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:54.274809 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:54.275420 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:54.275729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:54.276029 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4 admin admin] PATH_INFO: `/v3/projects/aeddf47fdd5146fea6246fa8260cef07` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:54.277135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeddf47fdd5146fea6246fa8260cef07)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:54.279020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:54.325041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeddf47fdd5146fea6246fa8260cef07', 'request_id': 'req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:54.332356 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeddf47fdd5146fea6246fa8260cef07', 'request_id': 'req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:54.338516 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeddf47fdd5146fea6246fa8260cef07', 'request_id': 'req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:54.348716 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4 admin admin] The token cache is being invalidate because project aeddf47fdd5146fea6246fa8260cef07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:24:54.349006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:54.349658 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:54.351145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1134/2265] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:24:54 2022] DELETE /identity/v3/projects/aeddf47fdd5146fea6246fa8260cef07 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:54.355088 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf82e8-4b90-43bc-96c8-569eec598f43 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:54.356698 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-03cf82e8-4b90-43bc-96c8-569eec598f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:54.370063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf82e8-4b90-43bc-96c8-569eec598f43 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:54.425580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc1484-db78-41a1-aeb4-f55b6ba254a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:54.426140 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc1484-db78-41a1-aeb4-f55b6ba254a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:54.426423 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc1484-db78-41a1-aeb4-f55b6ba254a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:54.426697 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc1484-db78-41a1-aeb4-f55b6ba254a8 admin admin] PATH_INFO: `/v3/users/8646611b2d834d01a7bac6fa0330ec59` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:54.428342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc1484-db78-41a1-aeb4-f55b6ba254a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8646611b2d834d01a7bac6fa0330ec59)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:54.430244 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc1484-db78-41a1-aeb4-f55b6ba254a8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:54.477107 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fafc1484-db78-41a1-aeb4-f55b6ba254a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8646611b2d834d01a7bac6fa0330ec59'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:54.484325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fafc1484-db78-41a1-aeb4-f55b6ba254a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8646611b2d834d01a7bac6fa0330ec59'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:54.485022 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fafc1484-db78-41a1-aeb4-f55b6ba254a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8646611b2d834d01a7bac6fa0330ec59'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:54.485633 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fafc1484-db78-41a1-aeb4-f55b6ba254a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8646611b2d834d01a7bac6fa0330ec59'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:54.493036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fafc1484-db78-41a1-aeb4-f55b6ba254a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8646611b2d834d01a7bac6fa0330ec59'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:54.510625 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1132/2266] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:24:54 2022] DELETE /identity/v3/users/8646611b2d834d01a7bac6fa0330ec59 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:24:54.954255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:54.956868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:54.972404 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca40b6-0e8c-4a7b-9e1d-3434eed4b7e4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:55.036543 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15124681-900e-42d9-8cad-d0368a68e887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:55.037293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15124681-900e-42d9-8cad-d0368a68e887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:55.037677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15124681-900e-42d9-8cad-d0368a68e887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:55.038022 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15124681-900e-42d9-8cad-d0368a68e887 admin admin] PATH_INFO: `/v3/projects/89a00b05b69942529ff2b4b95affabd6` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:55.039229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15124681-900e-42d9-8cad-d0368a68e887 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89a00b05b69942529ff2b4b95affabd6)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:55.041291 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15124681-900e-42d9-8cad-d0368a68e887 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:55.044114 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:55.088890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-15124681-900e-42d9-8cad-d0368a68e887 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89a00b05b69942529ff2b4b95affabd6', 'request_id': 'req-15124681-900e-42d9-8cad-d0368a68e887'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:55.096559 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-15124681-900e-42d9-8cad-d0368a68e887 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89a00b05b69942529ff2b4b95affabd6', 'request_id': 'req-15124681-900e-42d9-8cad-d0368a68e887'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:55.103433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-15124681-900e-42d9-8cad-d0368a68e887 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89a00b05b69942529ff2b4b95affabd6', 'request_id': 'req-15124681-900e-42d9-8cad-d0368a68e887'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:55.113825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-15124681-900e-42d9-8cad-d0368a68e887 admin admin] The token cache is being invalidate because project 89a00b05b69942529ff2b4b95affabd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:24:55.114115 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-15124681-900e-42d9-8cad-d0368a68e887 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:55.114764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-15124681-900e-42d9-8cad-d0368a68e887 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:55.116381 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1135/2267] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:24:54 2022] DELETE /identity/v3/projects/89a00b05b69942529ff2b4b95affabd6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:55.123331 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf3e99-159b-42ec-8b48-287c88777b34 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:55.123647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf3e99-159b-42ec-8b48-287c88777b34 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:55.124120 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf3e99-159b-42ec-8b48-287c88777b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:55.143413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-37cf3e99-159b-42ec-8b48-287c88777b34 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:55.144658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-37cf3e99-159b-42ec-8b48-287c88777b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:55.149315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:55.207398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1133/2268] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:55.214457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2056926-4ccf-43a6-ae3f-896d3d968aa5 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:55.214908 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2056926-4ccf-43a6-ae3f-896d3d968aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:55.215036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2056926-4ccf-43a6-ae3f-896d3d968aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:55.235492 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f2056926-4ccf-43a6-ae3f-896d3d968aa5 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:55.236712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f2056926-4ccf-43a6-ae3f-896d3d968aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:55.247660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1136/2269] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:55.249084 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf3e99-159b-42ec-8b48-287c88777b34 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:55.262564 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf3e99-159b-42ec-8b48-287c88777b34 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:55.267227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64cf7d-606a-4044-ba9e-b2878cb48eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:55.267866 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64cf7d-606a-4044-ba9e-b2878cb48eef admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:55.268148 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64cf7d-606a-4044-ba9e-b2878cb48eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:55.268423 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64cf7d-606a-4044-ba9e-b2878cb48eef admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:55.269064 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64cf7d-606a-4044-ba9e-b2878cb48eef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:24:55.269368 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64cf7d-606a-4044-ba9e-b2878cb48eef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:55.270962 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64cf7d-606a-4044-ba9e-b2878cb48eef admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:55.283174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1134/2270] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:24:55.288126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2056926-4ccf-43a6-ae3f-896d3d968aa5 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:55.297278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2056926-4ccf-43a6-ae3f-896d3d968aa5 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:55.301906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86d5c1-07ed-424f-a76f-1ebf0ae3c5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:55.302500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86d5c1-07ed-424f-a76f-1ebf0ae3c5ae admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:55.302823 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86d5c1-07ed-424f-a76f-1ebf0ae3c5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:55.303096 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86d5c1-07ed-424f-a76f-1ebf0ae3c5ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:55.303754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86d5c1-07ed-424f-a76f-1ebf0ae3c5ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:55.305649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86d5c1-07ed-424f-a76f-1ebf0ae3c5ae admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:55.331794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1137/2271] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:24:55 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:55.336423 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64cf7d-606a-4044-ba9e-b2878cb48eef admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:55.346310 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64cf7d-606a-4044-ba9e-b2878cb48eef admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:55.400794 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f4eeb-64d0-41b3-98c8-477786d3e200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:55.401394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f4eeb-64d0-41b3-98c8-477786d3e200 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:55.401672 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f4eeb-64d0-41b3-98c8-477786d3e200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:55.401940 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f4eeb-64d0-41b3-98c8-477786d3e200 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:55.402578 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f4eeb-64d0-41b3-98c8-477786d3e200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:55.404518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f4eeb-64d0-41b3-98c8-477786d3e200 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:55.426060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1135/2272] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:24:55 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:55.430266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86d5c1-07ed-424f-a76f-1ebf0ae3c5ae admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:55.440541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86d5c1-07ed-424f-a76f-1ebf0ae3c5ae admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:55.445619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10489de-2da5-4834-ab0f-ab79f9732cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:55.446221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10489de-2da5-4834-ab0f-ab79f9732cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:55.446508 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10489de-2da5-4834-ab0f-ab79f9732cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:55.446784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10489de-2da5-4834-ab0f-ab79f9732cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:55.447430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10489de-2da5-4834-ab0f-ab79f9732cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:55.449057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10489de-2da5-4834-ab0f-ab79f9732cb9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:55.459558 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1138/2273] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:55.463359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f4eeb-64d0-41b3-98c8-477786d3e200 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:55.473364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f4eeb-64d0-41b3-98c8-477786d3e200 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:55.478077 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8216e-da74-4aa6-8122-48ebab284edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:55.478673 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8216e-da74-4aa6-8122-48ebab284edb admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:55.478955 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8216e-da74-4aa6-8122-48ebab284edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:55.479228 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8216e-da74-4aa6-8122-48ebab284edb admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:55.479909 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8216e-da74-4aa6-8122-48ebab284edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:55.481494 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8216e-da74-4aa6-8122-48ebab284edb admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:55.493018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1136/2274] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:55.496976 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10489de-2da5-4834-ab0f-ab79f9732cb9 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:55.507365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10489de-2da5-4834-ab0f-ab79f9732cb9 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:55.512768 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7b8e2-f30e-4b30-b36c-d50d37d298ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:55.513420 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7b8e2-f30e-4b30-b36c-d50d37d298ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:55.513730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7b8e2-f30e-4b30-b36c-d50d37d298ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:55.514009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7b8e2-f30e-4b30-b36c-d50d37d298ec admin admin] PATH_INFO: `/v3/projects/5d4a1c06ea1740b3bccb4616f0902309/users/e6035ab114bc423d97826940f9831ca7/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:55.529282 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7b8e2-f30e-4b30-b36c-d50d37d298ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d4a1c06ea1740b3bccb4616f0902309, user_id=e6035ab114bc423d97826940f9831ca7, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:55.532285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7b8e2-f30e-4b30-b36c-d50d37d298ec admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:55.544839 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1139/2275] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:24:55 2022] PUT /identity/v3/projects/5d4a1c06ea1740b3bccb4616f0902309/users/e6035ab114bc423d97826940f9831ca7/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:55.557048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38ad94-984d-4e2d-8889-1cef66724c4f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:55.557340 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38ad94-984d-4e2d-8889-1cef66724c4f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:55.557619 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38ad94-984d-4e2d-8889-1cef66724c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:55.585134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ec38ad94-984d-4e2d-8889-1cef66724c4f None None] MFA Rules not processed for user `e6035ab114bc423d97826940f9831ca7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:55.586353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ec38ad94-984d-4e2d-8889-1cef66724c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:55.649453 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1137/2276] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:55.653950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7b8e2-f30e-4b30-b36c-d50d37d298ec admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:55.713937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7b8e2-f30e-4b30-b36c-d50d37d298ec admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:55.718524 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ff5d0-10d2-489d-91b5-741599ed3d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:55.719124 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ff5d0-10d2-489d-91b5-741599ed3d94 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:55.719405 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ff5d0-10d2-489d-91b5-741599ed3d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:55.719712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ff5d0-10d2-489d-91b5-741599ed3d94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:55.720346 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ff5d0-10d2-489d-91b5-741599ed3d94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:55.722235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ff5d0-10d2-489d-91b5-741599ed3d94 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:55.748722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1140/2277] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:24:55 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:55.752843 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38ad94-984d-4e2d-8889-1cef66724c4f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:55.759528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:55.764118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38ad94-984d-4e2d-8889-1cef66724c4f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:55.819449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fa1f5-7ef4-4ed8-af41-02e5d30b29a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:55.820077 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fa1f5-7ef4-4ed8-af41-02e5d30b29a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:55.820354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fa1f5-7ef4-4ed8-af41-02e5d30b29a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:55.820622 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fa1f5-7ef4-4ed8-af41-02e5d30b29a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:55.821237 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fa1f5-7ef4-4ed8-af41-02e5d30b29a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:55.823126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fa1f5-7ef4-4ed8-af41-02e5d30b29a7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:55.843740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1138/2278] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:24:55 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:55.847647 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ff5d0-10d2-489d-91b5-741599ed3d94 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:55.858229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ff5d0-10d2-489d-91b5-741599ed3d94 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:55.862756 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d97fe3-656b-469a-be7c-3cc35fe40d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:55.863345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d97fe3-656b-469a-be7c-3cc35fe40d5b admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:55.863825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d97fe3-656b-469a-be7c-3cc35fe40d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:55.864102 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d97fe3-656b-469a-be7c-3cc35fe40d5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:55.864731 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d97fe3-656b-469a-be7c-3cc35fe40d5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:55.866852 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d97fe3-656b-469a-be7c-3cc35fe40d5b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:55.878268 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1141/2279] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:24:55.883354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fa1f5-7ef4-4ed8-af41-02e5d30b29a7 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:55.894413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fa1f5-7ef4-4ed8-af41-02e5d30b29a7 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:55.899475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b2ed6-6ff0-4a4c-9990-96b0c3ad67de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:55.900176 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b2ed6-6ff0-4a4c-9990-96b0c3ad67de admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:55.900456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b2ed6-6ff0-4a4c-9990-96b0c3ad67de admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:55.900731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b2ed6-6ff0-4a4c-9990-96b0c3ad67de admin admin] PATH_INFO: `/v3/projects/5964945fdd464ffb819314fdb5be0fa1/users/f8e6db3a9bcf4c7bba18818002280734/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:55.916370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b2ed6-6ff0-4a4c-9990-96b0c3ad67de admin admin] RBAC: Authorizing `identity:create_grant(project_id=5964945fdd464ffb819314fdb5be0fa1, user_id=f8e6db3a9bcf4c7bba18818002280734, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:55.919030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b2ed6-6ff0-4a4c-9990-96b0c3ad67de admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:55.947656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1139/2280] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:24:55 2022] PUT /identity/v3/projects/5964945fdd464ffb819314fdb5be0fa1/users/f8e6db3a9bcf4c7bba18818002280734/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:55.948805 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d97fe3-656b-469a-be7c-3cc35fe40d5b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:55.955742 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:55.979221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d97fe3-656b-469a-be7c-3cc35fe40d5b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:56.066777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2a6fc-87b3-47f0-bf90-7a6be62a0e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:56.067634 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2a6fc-87b3-47f0-bf90-7a6be62a0e95 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:56.067919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2a6fc-87b3-47f0-bf90-7a6be62a0e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:56.068252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2a6fc-87b3-47f0-bf90-7a6be62a0e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:56.069423 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2a6fc-87b3-47f0-bf90-7a6be62a0e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:56.071031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2a6fc-87b3-47f0-bf90-7a6be62a0e95 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:56.085969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1142/2281] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:56.087345 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b2ed6-6ff0-4a4c-9990-96b0c3ad67de admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:56.104725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b2ed6-6ff0-4a4c-9990-96b0c3ad67de admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:56.110225 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7dbaa-2cb6-4fa5-aeed-3d34fd6b6254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:56.111326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7dbaa-2cb6-4fa5-aeed-3d34fd6b6254 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:56.111481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7dbaa-2cb6-4fa5-aeed-3d34fd6b6254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:56.111779 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7dbaa-2cb6-4fa5-aeed-3d34fd6b6254 admin admin] PATH_INFO: `/v3/projects/5964945fdd464ffb819314fdb5be0fa1/users/f8e6db3a9bcf4c7bba18818002280734/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:56.113746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b7dbaa-2cb6-4fa5-aeed-3d34fd6b6254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5964945fdd464ffb819314fdb5be0fa1, user_id=f8e6db3a9bcf4c7bba18818002280734, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:56.117293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b7dbaa-2cb6-4fa5-aeed-3d34fd6b6254 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:56.134660 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1140/2282] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:24:56 2022] PUT /identity/v3/projects/5964945fdd464ffb819314fdb5be0fa1/users/f8e6db3a9bcf4c7bba18818002280734/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:56.146764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0a08b-a086-4f01-9ab0-56204f7d64cf None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:56.147163 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0a08b-a086-4f01-9ab0-56204f7d64cf None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:56.147906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0a08b-a086-4f01-9ab0-56204f7d64cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:56.180289 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-79f0a08b-a086-4f01-9ab0-56204f7d64cf None None] MFA Rules not processed for user `f8e6db3a9bcf4c7bba18818002280734`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:56.181670 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-79f0a08b-a086-4f01-9ab0-56204f7d64cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:56.262040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1143/2283] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:56 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:56.273562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7dbaa-2cb6-4fa5-aeed-3d34fd6b6254 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:56.275335 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-23b7dbaa-2cb6-4fa5-aeed-3d34fd6b6254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:56.332645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7dbaa-2cb6-4fa5-aeed-3d34fd6b6254 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:56.336894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700e63b-f5f9-4f0e-a40e-7609f3e02d04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:56.337493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700e63b-f5f9-4f0e-a40e-7609f3e02d04 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:56.337773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700e63b-f5f9-4f0e-a40e-7609f3e02d04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:56.338046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700e63b-f5f9-4f0e-a40e-7609f3e02d04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:56.347239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700e63b-f5f9-4f0e-a40e-7609f3e02d04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:56.348710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700e63b-f5f9-4f0e-a40e-7609f3e02d04 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:56.358304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1141/2284] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:56 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:56.924640 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0a08b-a086-4f01-9ab0-56204f7d64cf None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:56.989258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0a08b-a086-4f01-9ab0-56204f7d64cf None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:56.994214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de7f96-8b73-4664-9367-022f34680cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:56.994854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de7f96-8b73-4664-9367-022f34680cd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:56.995130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de7f96-8b73-4664-9367-022f34680cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:56.995397 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de7f96-8b73-4664-9367-022f34680cd9 admin admin] PATH_INFO: `/v3/users/e6035ab114bc423d97826940f9831ca7` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:56.996552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de7f96-8b73-4664-9367-022f34680cd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6035ab114bc423d97826940f9831ca7)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:56.998474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de7f96-8b73-4664-9367-022f34680cd9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:57.030841 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:57.056594 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-77de7f96-8b73-4664-9367-022f34680cd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6035ab114bc423d97826940f9831ca7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:57.063698 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-77de7f96-8b73-4664-9367-022f34680cd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6035ab114bc423d97826940f9831ca7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:57.064405 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-77de7f96-8b73-4664-9367-022f34680cd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6035ab114bc423d97826940f9831ca7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:57.065034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-77de7f96-8b73-4664-9367-022f34680cd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6035ab114bc423d97826940f9831ca7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:57.075298 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-77de7f96-8b73-4664-9367-022f34680cd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6035ab114bc423d97826940f9831ca7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:57.093180 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1144/2285] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:24:56 2022] DELETE /identity/v3/users/e6035ab114bc423d97826940f9831ca7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:24:57.097665 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c262fb7-c763-4643-ab85-bf48fcd56f11 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:57.098133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c262fb7-c763-4643-ab85-bf48fcd56f11 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:57.098270 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c262fb7-c763-4643-ab85-bf48fcd56f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:57.117970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8c262fb7-c763-4643-ab85-bf48fcd56f11 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:57.119173 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8c262fb7-c763-4643-ab85-bf48fcd56f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:57.171291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:57.183622 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1142/2286] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:57.193424 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de7f96-8b73-4664-9367-022f34680cd9 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:57.194887 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-77de7f96-8b73-4664-9367-022f34680cd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:57.264535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de7f96-8b73-4664-9367-022f34680cd9 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:57.267618 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37630101-f1be-42f9-a953-ab8b18c49133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:57.268400 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37630101-f1be-42f9-a953-ab8b18c49133 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:57.268691 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37630101-f1be-42f9-a953-ab8b18c49133 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:57.268967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37630101-f1be-42f9-a953-ab8b18c49133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:57.281031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37630101-f1be-42f9-a953-ab8b18c49133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:57.283179 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37630101-f1be-42f9-a953-ab8b18c49133 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:57.294697 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1145/2287] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:24:57.611635 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c262fb7-c763-4643-ab85-bf48fcd56f11 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:57.613327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8c262fb7-c763-4643-ab85-bf48fcd56f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:57.629911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c262fb7-c763-4643-ab85-bf48fcd56f11 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:57.634952 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a361ca6-73fe-459a-a4a7-1c0a467d7271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:57.635664 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a361ca6-73fe-459a-a4a7-1c0a467d7271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:57.635930 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a361ca6-73fe-459a-a4a7-1c0a467d7271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:57.636202 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a361ca6-73fe-459a-a4a7-1c0a467d7271 admin admin] PATH_INFO: `/v3/projects/5d4a1c06ea1740b3bccb4616f0902309` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:57.637386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a361ca6-73fe-459a-a4a7-1c0a467d7271 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d4a1c06ea1740b3bccb4616f0902309)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:57.639334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a361ca6-73fe-459a-a4a7-1c0a467d7271 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:57.692485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0a361ca6-73fe-459a-a4a7-1c0a467d7271 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d4a1c06ea1740b3bccb4616f0902309', 'request_id': 'req-0a361ca6-73fe-459a-a4a7-1c0a467d7271'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:57.699755 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0a361ca6-73fe-459a-a4a7-1c0a467d7271 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d4a1c06ea1740b3bccb4616f0902309', 'request_id': 'req-0a361ca6-73fe-459a-a4a7-1c0a467d7271'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:57.706125 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0a361ca6-73fe-459a-a4a7-1c0a467d7271 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d4a1c06ea1740b3bccb4616f0902309', 'request_id': 'req-0a361ca6-73fe-459a-a4a7-1c0a467d7271'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:57.717261 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0a361ca6-73fe-459a-a4a7-1c0a467d7271 admin admin] The token cache is being invalidate because project 5d4a1c06ea1740b3bccb4616f0902309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:24:57.717553 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0a361ca6-73fe-459a-a4a7-1c0a467d7271 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:57.718210 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0a361ca6-73fe-459a-a4a7-1c0a467d7271 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:57.719756 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1143/2288] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:24:57 2022] DELETE /identity/v3/projects/5d4a1c06ea1740b3bccb4616f0902309 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:57.723799 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37630101-f1be-42f9-a953-ab8b18c49133 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:57.733944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37630101-f1be-42f9-a953-ab8b18c49133 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:57.788561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b1290-0b0c-410b-8541-2fe42f07bc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:57.789211 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b1290-0b0c-410b-8541-2fe42f07bc44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:57.789473 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b1290-0b0c-410b-8541-2fe42f07bc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:57.789741 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b1290-0b0c-410b-8541-2fe42f07bc44 admin admin] PATH_INFO: `/v3/users/f8e6db3a9bcf4c7bba18818002280734` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:57.790838 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b1290-0b0c-410b-8541-2fe42f07bc44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8e6db3a9bcf4c7bba18818002280734)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:57.792773 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b1290-0b0c-410b-8541-2fe42f07bc44 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:57.838024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9e2b1290-0b0c-410b-8541-2fe42f07bc44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8e6db3a9bcf4c7bba18818002280734'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:57.839187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:57.846575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9e2b1290-0b0c-410b-8541-2fe42f07bc44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8e6db3a9bcf4c7bba18818002280734'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:57.847254 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9e2b1290-0b0c-410b-8541-2fe42f07bc44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8e6db3a9bcf4c7bba18818002280734'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:57.847990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9e2b1290-0b0c-410b-8541-2fe42f07bc44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8e6db3a9bcf4c7bba18818002280734'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:57.856083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9e2b1290-0b0c-410b-8541-2fe42f07bc44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8e6db3a9bcf4c7bba18818002280734'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:57.873768 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1146/2289] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:24:57 2022] DELETE /identity/v3/users/f8e6db3a9bcf4c7bba18818002280734 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:58.153326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a361ca6-73fe-459a-a4a7-1c0a467d7271 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:58.154800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0a361ca6-73fe-459a-a4a7-1c0a467d7271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:58.169441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a361ca6-73fe-459a-a4a7-1c0a467d7271 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:58.231049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037515b6-4548-40b1-bedc-ffc17dc41df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:58.231721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037515b6-4548-40b1-bedc-ffc17dc41df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:58.232008 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037515b6-4548-40b1-bedc-ffc17dc41df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:58.232285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037515b6-4548-40b1-bedc-ffc17dc41df1 admin admin] PATH_INFO: `/v3/projects/5964945fdd464ffb819314fdb5be0fa1` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:58.233414 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037515b6-4548-40b1-bedc-ffc17dc41df1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5964945fdd464ffb819314fdb5be0fa1)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:58.235332 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037515b6-4548-40b1-bedc-ffc17dc41df1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:58.286662 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-037515b6-4548-40b1-bedc-ffc17dc41df1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5964945fdd464ffb819314fdb5be0fa1', 'request_id': 'req-037515b6-4548-40b1-bedc-ffc17dc41df1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:58.295576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-037515b6-4548-40b1-bedc-ffc17dc41df1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5964945fdd464ffb819314fdb5be0fa1', 'request_id': 'req-037515b6-4548-40b1-bedc-ffc17dc41df1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:58.302705 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-037515b6-4548-40b1-bedc-ffc17dc41df1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5964945fdd464ffb819314fdb5be0fa1', 'request_id': 'req-037515b6-4548-40b1-bedc-ffc17dc41df1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:58.313423 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-037515b6-4548-40b1-bedc-ffc17dc41df1 admin admin] The token cache is being invalidate because project 5964945fdd464ffb819314fdb5be0fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:24:58.314000 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-037515b6-4548-40b1-bedc-ffc17dc41df1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:58.315145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-037515b6-4548-40b1-bedc-ffc17dc41df1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:24:58.317217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1144/2290] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:24:58 2022] DELETE /identity/v3/projects/5964945fdd464ffb819314fdb5be0fa1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:58.322139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d142dc-82e4-4358-8129-6a1d2fc0eacf None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:58.322620 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d142dc-82e4-4358-8129-6a1d2fc0eacf None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:58.322620 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d142dc-82e4-4358-8129-6a1d2fc0eacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:58.342743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-85d142dc-82e4-4358-8129-6a1d2fc0eacf None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:58.343978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-85d142dc-82e4-4358-8129-6a1d2fc0eacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:58.402672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1147/2291] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:58.410988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0166b7a-cb89-4ac3-aa0d-50ed69f13619 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:58.411320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0166b7a-cb89-4ac3-aa0d-50ed69f13619 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:58.411590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0166b7a-cb89-4ac3-aa0d-50ed69f13619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:58.446041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d0166b7a-cb89-4ac3-aa0d-50ed69f13619 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:58.447267 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d0166b7a-cb89-4ac3-aa0d-50ed69f13619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:58.459388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1145/2292] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:58.460651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d142dc-82e4-4358-8129-6a1d2fc0eacf None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:58.470285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d142dc-82e4-4358-8129-6a1d2fc0eacf None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:58.476421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d3692-397d-4096-be40-9aa1e81b895e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:58.476421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d3692-397d-4096-be40-9aa1e81b895e admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:58.476421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d3692-397d-4096-be40-9aa1e81b895e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:58.476421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d3692-397d-4096-be40-9aa1e81b895e admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:58.476948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d3692-397d-4096-be40-9aa1e81b895e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:24:58.476948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d3692-397d-4096-be40-9aa1e81b895e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:58.478547 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d3692-397d-4096-be40-9aa1e81b895e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:58.491545 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1148/2293] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:24:58.501178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0166b7a-cb89-4ac3-aa0d-50ed69f13619 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:58.511867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0166b7a-cb89-4ac3-aa0d-50ed69f13619 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:58.519573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5297ec-4823-4411-8faf-43214f653d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:58.520206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5297ec-4823-4411-8faf-43214f653d8a admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:58.520490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5297ec-4823-4411-8faf-43214f653d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:58.520769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5297ec-4823-4411-8faf-43214f653d8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:58.522136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5297ec-4823-4411-8faf-43214f653d8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:58.525414 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5297ec-4823-4411-8faf-43214f653d8a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:58.530507 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:58.556513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1146/2294] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:24:58 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:24:58.560910 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d3692-397d-4096-be40-9aa1e81b895e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:58.572641 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d3692-397d-4096-be40-9aa1e81b895e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:58.644677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37122cc0-f697-48a7-9ead-a54cf37e9222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:58.645292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37122cc0-f697-48a7-9ead-a54cf37e9222 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:58.645587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37122cc0-f697-48a7-9ead-a54cf37e9222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:58.645857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37122cc0-f697-48a7-9ead-a54cf37e9222 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:58.647065 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37122cc0-f697-48a7-9ead-a54cf37e9222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:58.649003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37122cc0-f697-48a7-9ead-a54cf37e9222 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:58.679651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1149/2295] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:24:58 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:24:58.680396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5297ec-4823-4411-8faf-43214f653d8a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:58.691083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5297ec-4823-4411-8faf-43214f653d8a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:58.696403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfaedf-29af-4a85-a833-d4dd747374e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:58.697007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfaedf-29af-4a85-a833-d4dd747374e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:58.697287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfaedf-29af-4a85-a833-d4dd747374e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:58.697560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfaedf-29af-4a85-a833-d4dd747374e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:58.698201 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfaedf-29af-4a85-a833-d4dd747374e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:58.699969 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfaedf-29af-4a85-a833-d4dd747374e1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:58.710525 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1147/2296] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:58.714703 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37122cc0-f697-48a7-9ead-a54cf37e9222 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:58.725344 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37122cc0-f697-48a7-9ead-a54cf37e9222 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:58.730169 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820de97-6156-467f-981f-7a88e661ccd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:58.730771 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820de97-6156-467f-981f-7a88e661ccd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:58.731049 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820de97-6156-467f-981f-7a88e661ccd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:58.731338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820de97-6156-467f-981f-7a88e661ccd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:58.732014 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820de97-6156-467f-981f-7a88e661ccd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:58.736774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820de97-6156-467f-981f-7a88e661ccd3 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:58.747698 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1150/2297] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:58.751667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfaedf-29af-4a85-a833-d4dd747374e1 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:58.762138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfaedf-29af-4a85-a833-d4dd747374e1 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:58.766781 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b701772-66ee-407d-9373-afbfced4e71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:58.767426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b701772-66ee-407d-9373-afbfced4e71f admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:58.767744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b701772-66ee-407d-9373-afbfced4e71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:58.768018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b701772-66ee-407d-9373-afbfced4e71f admin admin] PATH_INFO: `/v3/projects/685bb11a28794448b00df7ff778f1eff/users/413a44abfed9477fb8b516ddc7e44ef1/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:58.793912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b701772-66ee-407d-9373-afbfced4e71f admin admin] RBAC: Authorizing `identity:create_grant(project_id=685bb11a28794448b00df7ff778f1eff, user_id=413a44abfed9477fb8b516ddc7e44ef1, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:58.796968 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b701772-66ee-407d-9373-afbfced4e71f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:58.814610 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1148/2298] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:24:58 2022] PUT /identity/v3/projects/685bb11a28794448b00df7ff778f1eff/users/413a44abfed9477fb8b516ddc7e44ef1/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:24:58.833227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd49cf0-37c1-479d-833d-03dbfbdfe9ca None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:58.833905 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd49cf0-37c1-479d-833d-03dbfbdfe9ca None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:58.834518 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd49cf0-37c1-479d-833d-03dbfbdfe9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:58.878338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-4cd49cf0-37c1-479d-833d-03dbfbdfe9ca None None] MFA Rules not processed for user `413a44abfed9477fb8b516ddc7e44ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:58.879608 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4cd49cf0-37c1-479d-833d-03dbfbdfe9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:58.904070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:58.961605 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1151/2299] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:24:58.962851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b701772-66ee-407d-9373-afbfced4e71f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:59.041631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b701772-66ee-407d-9373-afbfced4e71f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:59.047390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb38c39-ce3b-4832-a984-41e69a4fd6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:59.048065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb38c39-ce3b-4832-a984-41e69a4fd6f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:59.048526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb38c39-ce3b-4832-a984-41e69a4fd6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:59.048667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb38c39-ce3b-4832-a984-41e69a4fd6f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:59.049260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb38c39-ce3b-4832-a984-41e69a4fd6f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:59.051215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb38c39-ce3b-4832-a984-41e69a4fd6f4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:59.089194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1149/2300] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:24:58 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:59.093361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd49cf0-37c1-479d-833d-03dbfbdfe9ca None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:59.108706 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd49cf0-37c1-479d-833d-03dbfbdfe9ca None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:59.200021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f0ac1-d529-43da-8130-7e5fe0b4dff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:59.201425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f0ac1-d529-43da-8130-7e5fe0b4dff0 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:59.203210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f0ac1-d529-43da-8130-7e5fe0b4dff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:59.203750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f0ac1-d529-43da-8130-7e5fe0b4dff0 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:59.204622 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f0ac1-d529-43da-8130-7e5fe0b4dff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:59.206801 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f0ac1-d529-43da-8130-7e5fe0b4dff0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:59.234507 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1152/2301] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:24:59 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:59.243527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb38c39-ce3b-4832-a984-41e69a4fd6f4 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:59.252851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb38c39-ce3b-4832-a984-41e69a4fd6f4 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:59.259291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30d081-75cd-497f-a490-afaccde6779b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:59.259603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30d081-75cd-497f-a490-afaccde6779b admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:59.259891 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30d081-75cd-497f-a490-afaccde6779b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:59.260169 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30d081-75cd-497f-a490-afaccde6779b admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:59.260829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30d081-75cd-497f-a490-afaccde6779b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:59.262678 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30d081-75cd-497f-a490-afaccde6779b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:59.278382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1150/2302] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:59.282581 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f0ac1-d529-43da-8130-7e5fe0b4dff0 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:59.293727 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f0ac1-d529-43da-8130-7e5fe0b4dff0 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:59.298610 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40146837-9b53-4743-b638-3039d9884ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:59.299281 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40146837-9b53-4743-b638-3039d9884ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:59.299664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40146837-9b53-4743-b638-3039d9884ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:59.300004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40146837-9b53-4743-b638-3039d9884ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:59.300716 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40146837-9b53-4743-b638-3039d9884ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:59.302406 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40146837-9b53-4743-b638-3039d9884ab5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:59.312479 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1153/2303] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:59.316630 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30d081-75cd-497f-a490-afaccde6779b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:59.327358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30d081-75cd-497f-a490-afaccde6779b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:59.333100 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e45d0a-d651-40b6-8661-bd14bc677cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:59.334322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e45d0a-d651-40b6-8661-bd14bc677cfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:59.334600 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e45d0a-d651-40b6-8661-bd14bc677cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:59.334870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e45d0a-d651-40b6-8661-bd14bc677cfa admin admin] PATH_INFO: `/v3/projects/fe37a41632b74f029c60d97626d690d0/users/5d98ce2356a14a8fb0237e89e4d42842/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:59.350387 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e45d0a-d651-40b6-8661-bd14bc677cfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe37a41632b74f029c60d97626d690d0, user_id=5d98ce2356a14a8fb0237e89e4d42842, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:59.353016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e45d0a-d651-40b6-8661-bd14bc677cfa admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:59.365582 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1151/2304] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:24:59 2022] PUT /identity/v3/projects/fe37a41632b74f029c60d97626d690d0/users/5d98ce2356a14a8fb0237e89e4d42842/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:59.375771 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de63d3b-ec60-49ac-a898-9aee9ad9def2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:59.376276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de63d3b-ec60-49ac-a898-9aee9ad9def2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:59.376446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de63d3b-ec60-49ac-a898-9aee9ad9def2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:59.405136 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-2de63d3b-ec60-49ac-a898-9aee9ad9def2 None None] MFA Rules not processed for user `5d98ce2356a14a8fb0237e89e4d42842`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:24:59.406412 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2de63d3b-ec60-49ac-a898-9aee9ad9def2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:24:59.487167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1154/2305] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:24:59.491615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e45d0a-d651-40b6-8661-bd14bc677cfa admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:59.559685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e45d0a-d651-40b6-8661-bd14bc677cfa admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:59.564440 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b54fe9-3876-4ed8-b21c-1b1f5d1e1803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:59.565053 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b54fe9-3876-4ed8-b21c-1b1f5d1e1803 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:59.565333 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b54fe9-3876-4ed8-b21c-1b1f5d1e1803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:59.565609 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b54fe9-3876-4ed8-b21c-1b1f5d1e1803 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:59.566255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b54fe9-3876-4ed8-b21c-1b1f5d1e1803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:59.568276 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b54fe9-3876-4ed8-b21c-1b1f5d1e1803 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:59.575148 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:59.604811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1152/2306] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:24:59 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:59.607658 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de63d3b-ec60-49ac-a898-9aee9ad9def2 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:59.618675 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de63d3b-ec60-49ac-a898-9aee9ad9def2 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:59.649442 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:24:59.709040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e1455-614e-41ea-bca2-91c5da5cb5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:59.709871 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e1455-614e-41ea-bca2-91c5da5cb5d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:59.709989 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e1455-614e-41ea-bca2-91c5da5cb5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:59.710216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e1455-614e-41ea-bca2-91c5da5cb5d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:59.710862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e1455-614e-41ea-bca2-91c5da5cb5d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:59.713048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e1455-614e-41ea-bca2-91c5da5cb5d8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:59.743541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1155/2307] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:24:59 2022] POST /identity/v3/users => generated 295 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:24:59.746215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b54fe9-3876-4ed8-b21c-1b1f5d1e1803 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:59.757467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b54fe9-3876-4ed8-b21c-1b1f5d1e1803 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:59.762261 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57f25f-bea2-42fe-bb58-b35bcdb447b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:59.762876 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57f25f-bea2-42fe-bb58-b35bcdb447b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:59.763161 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57f25f-bea2-42fe-bb58-b35bcdb447b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:59.763440 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57f25f-bea2-42fe-bb58-b35bcdb447b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:59.764128 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57f25f-bea2-42fe-bb58-b35bcdb447b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:59.765754 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57f25f-bea2-42fe-bb58-b35bcdb447b0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:59.779545 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1153/2308] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:59.781560 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e1455-614e-41ea-bca2-91c5da5cb5d8 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:59.792337 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e1455-614e-41ea-bca2-91c5da5cb5d8 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:59.797973 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c2389-928e-495f-aecd-08ae731f05ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:59.798815 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c2389-928e-495f-aecd-08ae731f05ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:59.799109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c2389-928e-495f-aecd-08ae731f05ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:59.799388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c2389-928e-495f-aecd-08ae731f05ea admin admin] PATH_INFO: `/v3/projects/8256294cc2c3490a99f41a450c3eb012/users/68088f88af5646c991ce8c571663713f/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:59.818810 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c2389-928e-495f-aecd-08ae731f05ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8256294cc2c3490a99f41a450c3eb012, user_id=68088f88af5646c991ce8c571663713f, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:59.821506 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c2389-928e-495f-aecd-08ae731f05ea admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:59.837674 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1156/2309] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:24:59 2022] PUT /identity/v3/projects/8256294cc2c3490a99f41a450c3eb012/users/68088f88af5646c991ce8c571663713f/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:59.841566 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57f25f-bea2-42fe-bb58-b35bcdb447b0 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:59.853103 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57f25f-bea2-42fe-bb58-b35bcdb447b0 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:59.920270 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d064d-6f06-42d4-b018-0f6850403e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:59.920637 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d064d-6f06-42d4-b018-0f6850403e7e admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:59.920996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d064d-6f06-42d4-b018-0f6850403e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:59.921365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d064d-6f06-42d4-b018-0f6850403e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:59.922831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d064d-6f06-42d4-b018-0f6850403e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:59.923951 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d064d-6f06-42d4-b018-0f6850403e7e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:59.937521 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1154/2310] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:24:59.941966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c2389-928e-495f-aecd-08ae731f05ea admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:24:59.953755 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c2389-928e-495f-aecd-08ae731f05ea admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:24:59.959225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01176a-0dcd-453d-8667-9dcab11a14e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:24:59.959923 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01176a-0dcd-453d-8667-9dcab11a14e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:59.960207 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01176a-0dcd-453d-8667-9dcab11a14e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:59.960477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01176a-0dcd-453d-8667-9dcab11a14e6 admin admin] PATH_INFO: `/v3/projects/8256294cc2c3490a99f41a450c3eb012/users/68088f88af5646c991ce8c571663713f/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:24:59.962505 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01176a-0dcd-453d-8667-9dcab11a14e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8256294cc2c3490a99f41a450c3eb012, user_id=68088f88af5646c991ce8c571663713f, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:24:59.965234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01176a-0dcd-453d-8667-9dcab11a14e6 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:24:59.977795 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1157/2311] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:24:59 2022] PUT /identity/v3/projects/8256294cc2c3490a99f41a450c3eb012/users/68088f88af5646c991ce8c571663713f/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:24:59.989493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3631064-fcf2-460a-8829-d5b24d1f4a9d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:24:59.989493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3631064-fcf2-460a-8829-d5b24d1f4a9d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:24:59.989886 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3631064-fcf2-460a-8829-d5b24d1f4a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:00.022680 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c3631064-fcf2-460a-8829-d5b24d1f4a9d None None] MFA Rules not processed for user `68088f88af5646c991ce8c571663713f`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:00.036149 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c3631064-fcf2-460a-8829-d5b24d1f4a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:00.120128 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1155/2312] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:24:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:00.133184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01176a-0dcd-453d-8667-9dcab11a14e6 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:00.135564 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3e01176a-0dcd-453d-8667-9dcab11a14e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:00.207799 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01176a-0dcd-453d-8667-9dcab11a14e6 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:00.215013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f9fd3-3d4b-40f8-a7ac-b397d022392d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:00.215283 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f9fd3-3d4b-40f8-a7ac-b397d022392d service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:00.215576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f9fd3-3d4b-40f8-a7ac-b397d022392d service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:00.215887 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f9fd3-3d4b-40f8-a7ac-b397d022392d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:00.287088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f9fd3-3d4b-40f8-a7ac-b397d022392d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:00.289095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f9fd3-3d4b-40f8-a7ac-b397d022392d service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:00.299369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1158/2313] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:25:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:25:02.954434 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3631064-fcf2-460a-8829-d5b24d1f4a9d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:02.955860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c3631064-fcf2-460a-8829-d5b24d1f4a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:02.968971 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3631064-fcf2-460a-8829-d5b24d1f4a9d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:02.973015 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db5d54-4ffe-45fa-9d6c-fbb79fa67996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:02.973626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db5d54-4ffe-45fa-9d6c-fbb79fa67996 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:02.973906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db5d54-4ffe-45fa-9d6c-fbb79fa67996 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:02.974178 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db5d54-4ffe-45fa-9d6c-fbb79fa67996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:03.029036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:03.033049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db5d54-4ffe-45fa-9d6c-fbb79fa67996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:03.034580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db5d54-4ffe-45fa-9d6c-fbb79fa67996 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:03.046099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1156/2314] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:25:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:25:03.755157 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f9fd3-3d4b-40f8-a7ac-b397d022392d service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:03.764375 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f9fd3-3d4b-40f8-a7ac-b397d022392d service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:03.768137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be99d8-b55d-4f56-9833-5ffd5dc8af33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:03.768903 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be99d8-b55d-4f56-9833-5ffd5dc8af33 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:03.769185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be99d8-b55d-4f56-9833-5ffd5dc8af33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:03.769484 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be99d8-b55d-4f56-9833-5ffd5dc8af33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:03.779217 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be99d8-b55d-4f56-9833-5ffd5dc8af33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:03.780720 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be99d8-b55d-4f56-9833-5ffd5dc8af33 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:03.791882 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1159/2315] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:25:03 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:25:04.970343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db5d54-4ffe-45fa-9d6c-fbb79fa67996 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:04.971808 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-15db5d54-4ffe-45fa-9d6c-fbb79fa67996 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:05.042534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db5d54-4ffe-45fa-9d6c-fbb79fa67996 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:05.049366 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24337405-0a5d-4e46-b60d-a1982519ba70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:05.050644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24337405-0a5d-4e46-b60d-a1982519ba70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:05.050912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24337405-0a5d-4e46-b60d-a1982519ba70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:05.051255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24337405-0a5d-4e46-b60d-a1982519ba70 admin admin] PATH_INFO: `/v3/users/cf14dd15f92b474cb7608a6efb390fe9` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:05.053481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24337405-0a5d-4e46-b60d-a1982519ba70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf14dd15f92b474cb7608a6efb390fe9)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:05.055675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24337405-0a5d-4e46-b60d-a1982519ba70 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:05.109281 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-24337405-0a5d-4e46-b60d-a1982519ba70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf14dd15f92b474cb7608a6efb390fe9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:05.117506 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-24337405-0a5d-4e46-b60d-a1982519ba70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf14dd15f92b474cb7608a6efb390fe9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:05.118284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-24337405-0a5d-4e46-b60d-a1982519ba70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf14dd15f92b474cb7608a6efb390fe9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:05.119970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-24337405-0a5d-4e46-b60d-a1982519ba70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf14dd15f92b474cb7608a6efb390fe9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:05.132439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-24337405-0a5d-4e46-b60d-a1982519ba70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf14dd15f92b474cb7608a6efb390fe9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:05.162537 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1157/2316] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:25:04 2022] DELETE /identity/v3/users/cf14dd15f92b474cb7608a6efb390fe9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:05.165610 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b8b35-0acf-4c5a-91d8-6252f3a39e91 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:05.166212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b8b35-0acf-4c5a-91d8-6252f3a39e91 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:05.166212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b8b35-0acf-4c5a-91d8-6252f3a39e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:05.189413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d44b8b35-0acf-4c5a-91d8-6252f3a39e91 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:05.190764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d44b8b35-0acf-4c5a-91d8-6252f3a39e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:05.212589 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:05.273703 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1160/2317] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:05.284513 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24337405-0a5d-4e46-b60d-a1982519ba70 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:05.285958 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-24337405-0a5d-4e46-b60d-a1982519ba70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:05.367355 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24337405-0a5d-4e46-b60d-a1982519ba70 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:05.371420 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c7f6f-23c4-4e52-932c-e5d4c1eff2e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:05.372128 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c7f6f-23c4-4e52-932c-e5d4c1eff2e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:05.372557 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c7f6f-23c4-4e52-932c-e5d4c1eff2e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:05.372922 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c7f6f-23c4-4e52-932c-e5d4c1eff2e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:05.382832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c7f6f-23c4-4e52-932c-e5d4c1eff2e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:05.384477 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c7f6f-23c4-4e52-932c-e5d4c1eff2e1 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:05.394223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1158/2318] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:25:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:25:05.671632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b8b35-0acf-4c5a-91d8-6252f3a39e91 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:05.673118 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d44b8b35-0acf-4c5a-91d8-6252f3a39e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:05.689770 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b8b35-0acf-4c5a-91d8-6252f3a39e91 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:05.694528 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3f8ad-1123-417a-b37a-ba01693ff3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:05.695090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3f8ad-1123-417a-b37a-ba01693ff3c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:05.695369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3f8ad-1123-417a-b37a-ba01693ff3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:05.695675 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3f8ad-1123-417a-b37a-ba01693ff3c7 admin admin] PATH_INFO: `/v3/projects/e6669485f0024c83a7154101cfa48abc` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:05.696793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3f8ad-1123-417a-b37a-ba01693ff3c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6669485f0024c83a7154101cfa48abc)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:05.698710 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3f8ad-1123-417a-b37a-ba01693ff3c7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:05.747551 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-04e3f8ad-1123-417a-b37a-ba01693ff3c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6669485f0024c83a7154101cfa48abc', 'request_id': 'req-04e3f8ad-1123-417a-b37a-ba01693ff3c7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:05.755465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-04e3f8ad-1123-417a-b37a-ba01693ff3c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6669485f0024c83a7154101cfa48abc', 'request_id': 'req-04e3f8ad-1123-417a-b37a-ba01693ff3c7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:05.761862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-04e3f8ad-1123-417a-b37a-ba01693ff3c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6669485f0024c83a7154101cfa48abc', 'request_id': 'req-04e3f8ad-1123-417a-b37a-ba01693ff3c7'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:05.775175 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-04e3f8ad-1123-417a-b37a-ba01693ff3c7 admin admin] The token cache is being invalidate because project e6669485f0024c83a7154101cfa48abc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:25:05.775399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-04e3f8ad-1123-417a-b37a-ba01693ff3c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:05.776050 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-04e3f8ad-1123-417a-b37a-ba01693ff3c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:05.777679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1161/2319] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:25:05 2022] DELETE /identity/v3/projects/e6669485f0024c83a7154101cfa48abc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:05.782741 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225b035-d9f9-4aaa-b369-7729632f9fcb None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:05.783020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225b035-d9f9-4aaa-b369-7729632f9fcb None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:05.783285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225b035-d9f9-4aaa-b369-7729632f9fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:05.806063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-5225b035-d9f9-4aaa-b369-7729632f9fcb None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:05.807283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5225b035-d9f9-4aaa-b369-7729632f9fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:05.828140 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:05.895537 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1159/2320] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:05.903860 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63792447-745f-44e2-8aca-4ec200861b4b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:05.904143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63792447-745f-44e2-8aca-4ec200861b4b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:05.904417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63792447-745f-44e2-8aca-4ec200861b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:05.924680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-63792447-745f-44e2-8aca-4ec200861b4b None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:05.925884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-63792447-745f-44e2-8aca-4ec200861b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:05.933639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1162/2321] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:05.937852 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225b035-d9f9-4aaa-b369-7729632f9fcb None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:05.948074 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225b035-d9f9-4aaa-b369-7729632f9fcb None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:05.952642 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c368c-bc53-40b6-ab8f-ed4cc05d7aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:05.953244 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c368c-bc53-40b6-ab8f-ed4cc05d7aef admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:05.953519 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c368c-bc53-40b6-ab8f-ed4cc05d7aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:05.953788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c368c-bc53-40b6-ab8f-ed4cc05d7aef admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:05.954497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c368c-bc53-40b6-ab8f-ed4cc05d7aef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:25:05.954799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c368c-bc53-40b6-ab8f-ed4cc05d7aef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:05.956435 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c368c-bc53-40b6-ab8f-ed4cc05d7aef admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:05.969214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1160/2322] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:25:05.973517 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63792447-745f-44e2-8aca-4ec200861b4b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:05.982734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63792447-745f-44e2-8aca-4ec200861b4b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:05.987399 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a80ac-a27c-4b70-af33-404d8aa220b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:05.988028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a80ac-a27c-4b70-af33-404d8aa220b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:05.988310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a80ac-a27c-4b70-af33-404d8aa220b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:05.988583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a80ac-a27c-4b70-af33-404d8aa220b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:05.989241 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a80ac-a27c-4b70-af33-404d8aa220b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:05.991135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a80ac-a27c-4b70-af33-404d8aa220b1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:06.018654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1163/2323] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:25:05 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:25:06.023406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c368c-bc53-40b6-ab8f-ed4cc05d7aef admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:06.040097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c368c-bc53-40b6-ab8f-ed4cc05d7aef admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:06.101066 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4859f-5658-467a-9ef6-dc813584dd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:06.101702 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4859f-5658-467a-9ef6-dc813584dd25 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:06.101965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4859f-5658-467a-9ef6-dc813584dd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:06.102236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4859f-5658-467a-9ef6-dc813584dd25 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:06.102869 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4859f-5658-467a-9ef6-dc813584dd25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:06.104885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4859f-5658-467a-9ef6-dc813584dd25 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:06.126571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1161/2324] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:25:06 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:25:06.131335 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a80ac-a27c-4b70-af33-404d8aa220b1 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:06.142082 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a80ac-a27c-4b70-af33-404d8aa220b1 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:06.147171 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4e79a-0630-46ce-9e6a-299164398df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:06.147861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4e79a-0630-46ce-9e6a-299164398df3 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:06.148167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4e79a-0630-46ce-9e6a-299164398df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:06.148407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4e79a-0630-46ce-9e6a-299164398df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:06.149048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4e79a-0630-46ce-9e6a-299164398df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:06.150703 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4e79a-0630-46ce-9e6a-299164398df3 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:06.164400 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1164/2325] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:25:06.168392 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4859f-5658-467a-9ef6-dc813584dd25 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:06.179121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4859f-5658-467a-9ef6-dc813584dd25 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:06.183795 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566c276-f7a9-443c-8f72-e053aa92d4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:06.184391 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566c276-f7a9-443c-8f72-e053aa92d4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:06.184673 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566c276-f7a9-443c-8f72-e053aa92d4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:06.184947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566c276-f7a9-443c-8f72-e053aa92d4b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:06.185618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566c276-f7a9-443c-8f72-e053aa92d4b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:06.187207 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566c276-f7a9-443c-8f72-e053aa92d4b3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:06.198345 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1162/2326] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:25:06.202520 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4e79a-0630-46ce-9e6a-299164398df3 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:06.212725 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4e79a-0630-46ce-9e6a-299164398df3 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:06.217366 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ffc3b-4e02-4a79-bf62-9df5672cee61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:06.218048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ffc3b-4e02-4a79-bf62-9df5672cee61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:06.218328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ffc3b-4e02-4a79-bf62-9df5672cee61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:06.218602 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ffc3b-4e02-4a79-bf62-9df5672cee61 admin admin] PATH_INFO: `/v3/projects/c85b83d6b8e84306854ce0d9688e809b/users/868afe4c8aff4e2d87a8ddfb84550c41/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:06.233518 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ffc3b-4e02-4a79-bf62-9df5672cee61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c85b83d6b8e84306854ce0d9688e809b, user_id=868afe4c8aff4e2d87a8ddfb84550c41, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:06.236204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ffc3b-4e02-4a79-bf62-9df5672cee61 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:06.250182 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1165/2327] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:25:06 2022] PUT /identity/v3/projects/c85b83d6b8e84306854ce0d9688e809b/users/868afe4c8aff4e2d87a8ddfb84550c41/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:06.261904 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691d45b-f1ae-4c94-883a-e179922e05a8 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:06.262185 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691d45b-f1ae-4c94-883a-e179922e05a8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:06.262456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691d45b-f1ae-4c94-883a-e179922e05a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:06.291096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-0691d45b-f1ae-4c94-883a-e179922e05a8 None None] MFA Rules not processed for user `868afe4c8aff4e2d87a8ddfb84550c41`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:06.292263 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0691d45b-f1ae-4c94-883a-e179922e05a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:06.360037 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1163/2328] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:06.371397 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ffc3b-4e02-4a79-bf62-9df5672cee61 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:06.373281 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-044ffc3b-4e02-4a79-bf62-9df5672cee61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:06.424104 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:06.438685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ffc3b-4e02-4a79-bf62-9df5672cee61 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:06.444490 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00765613-e384-47db-8b5e-ec34825416ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:06.445129 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00765613-e384-47db-8b5e-ec34825416ca service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:06.445409 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00765613-e384-47db-8b5e-ec34825416ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:06.445682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00765613-e384-47db-8b5e-ec34825416ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:06.464913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00765613-e384-47db-8b5e-ec34825416ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:06.466214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00765613-e384-47db-8b5e-ec34825416ca service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:06.477642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1166/2329] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:25:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:25:06.504446 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691d45b-f1ae-4c94-883a-e179922e05a8 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:06.540156 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:06.606503 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691d45b-f1ae-4c94-883a-e179922e05a8 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:06.612633 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca0735-f4ee-4fd8-a9f8-7b9861f3c562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:06.616205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca0735-f4ee-4fd8-a9f8-7b9861f3c562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:06.616493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca0735-f4ee-4fd8-a9f8-7b9861f3c562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:06.616853 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca0735-f4ee-4fd8-a9f8-7b9861f3c562 admin admin] PATH_INFO: `/v3/users/868afe4c8aff4e2d87a8ddfb84550c41` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:06.618180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca0735-f4ee-4fd8-a9f8-7b9861f3c562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=868afe4c8aff4e2d87a8ddfb84550c41)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:06.620440 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca0735-f4ee-4fd8-a9f8-7b9861f3c562 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:06.694282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-76ca0735-f4ee-4fd8-a9f8-7b9861f3c562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '868afe4c8aff4e2d87a8ddfb84550c41'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:06.703530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-76ca0735-f4ee-4fd8-a9f8-7b9861f3c562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '868afe4c8aff4e2d87a8ddfb84550c41'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:06.717265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-76ca0735-f4ee-4fd8-a9f8-7b9861f3c562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '868afe4c8aff4e2d87a8ddfb84550c41'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:06.718970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-76ca0735-f4ee-4fd8-a9f8-7b9861f3c562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '868afe4c8aff4e2d87a8ddfb84550c41'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:06.740200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-76ca0735-f4ee-4fd8-a9f8-7b9861f3c562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '868afe4c8aff4e2d87a8ddfb84550c41'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:06.765815 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1164/2330] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:25:06 2022] DELETE /identity/v3/users/868afe4c8aff4e2d87a8ddfb84550c41 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:06.770561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a963d72-92a0-41c0-bb05-96e12ef4fa12 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:06.770856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a963d72-92a0-41c0-bb05-96e12ef4fa12 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:06.772235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a963d72-92a0-41c0-bb05-96e12ef4fa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:06.792362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8a963d72-92a0-41c0-bb05-96e12ef4fa12 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:06.793738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8a963d72-92a0-41c0-bb05-96e12ef4fa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:06.909675 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1167/2331] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:06.918205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca0735-f4ee-4fd8-a9f8-7b9861f3c562 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:06.919872 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-76ca0735-f4ee-4fd8-a9f8-7b9861f3c562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:06.999658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca0735-f4ee-4fd8-a9f8-7b9861f3c562 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:07.003805 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbca2ac-5c1f-4899-b52d-97033d722d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:07.004754 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbca2ac-5c1f-4899-b52d-97033d722d30 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:07.005035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbca2ac-5c1f-4899-b52d-97033d722d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:07.005343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbca2ac-5c1f-4899-b52d-97033d722d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:07.015778 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbca2ac-5c1f-4899-b52d-97033d722d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:07.017216 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbca2ac-5c1f-4899-b52d-97033d722d30 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:07.028655 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1165/2332] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:25:07.461632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a963d72-92a0-41c0-bb05-96e12ef4fa12 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:07.463701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8a963d72-92a0-41c0-bb05-96e12ef4fa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:07.478342 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a963d72-92a0-41c0-bb05-96e12ef4fa12 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:07.484512 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c572e-2a22-447a-9401-3dffbf0665e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:07.485526 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c572e-2a22-447a-9401-3dffbf0665e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:07.486104 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c572e-2a22-447a-9401-3dffbf0665e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:07.486564 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c572e-2a22-447a-9401-3dffbf0665e3 admin admin] PATH_INFO: `/v3/projects/c85b83d6b8e84306854ce0d9688e809b` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:07.487929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c572e-2a22-447a-9401-3dffbf0665e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c85b83d6b8e84306854ce0d9688e809b)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:07.490330 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c572e-2a22-447a-9401-3dffbf0665e3 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:07.551456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff8c572e-2a22-447a-9401-3dffbf0665e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c85b83d6b8e84306854ce0d9688e809b', 'request_id': 'req-ff8c572e-2a22-447a-9401-3dffbf0665e3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:07.560161 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff8c572e-2a22-447a-9401-3dffbf0665e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c85b83d6b8e84306854ce0d9688e809b', 'request_id': 'req-ff8c572e-2a22-447a-9401-3dffbf0665e3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:07.567413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff8c572e-2a22-447a-9401-3dffbf0665e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c85b83d6b8e84306854ce0d9688e809b', 'request_id': 'req-ff8c572e-2a22-447a-9401-3dffbf0665e3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:07.579917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff8c572e-2a22-447a-9401-3dffbf0665e3 admin admin] The token cache is being invalidate because project c85b83d6b8e84306854ce0d9688e809b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:25:07.579917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff8c572e-2a22-447a-9401-3dffbf0665e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:07.580597 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff8c572e-2a22-447a-9401-3dffbf0665e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:07.582386 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1168/2333] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:25:07 2022] DELETE /identity/v3/projects/c85b83d6b8e84306854ce0d9688e809b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:07.587046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee08a3e-cddc-4e28-af67-c80766265c64 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:07.587262 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee08a3e-cddc-4e28-af67-c80766265c64 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:07.587573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee08a3e-cddc-4e28-af67-c80766265c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:07.609917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-9ee08a3e-cddc-4e28-af67-c80766265c64 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:07.611172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9ee08a3e-cddc-4e28-af67-c80766265c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:07.641648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:07.678997 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1166/2334] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:07.687356 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adea58-c869-4071-b218-1eddefa75414 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:07.687726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adea58-c869-4071-b218-1eddefa75414 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:07.688001 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adea58-c869-4071-b218-1eddefa75414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:07.715168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c3adea58-c869-4071-b218-1eddefa75414 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:07.716414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c3adea58-c869-4071-b218-1eddefa75414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:07.728925 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1169/2335] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:07.730123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee08a3e-cddc-4e28-af67-c80766265c64 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:07.740948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee08a3e-cddc-4e28-af67-c80766265c64 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:07.747977 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68711784-23b4-4e0d-af84-1088b209d420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:07.748536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68711784-23b4-4e0d-af84-1088b209d420 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:07.748904 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68711784-23b4-4e0d-af84-1088b209d420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:07.749090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68711784-23b4-4e0d-af84-1088b209d420 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:07.749751 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68711784-23b4-4e0d-af84-1088b209d420 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:25:07.750059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68711784-23b4-4e0d-af84-1088b209d420 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:07.751965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68711784-23b4-4e0d-af84-1088b209d420 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:07.766820 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1167/2336] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:25:07.773185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3adea58-c869-4071-b218-1eddefa75414 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:07.782613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3adea58-c869-4071-b218-1eddefa75414 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:07.787796 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02ac9a-6c70-41ef-b74c-aa349b9c1cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:07.788394 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02ac9a-6c70-41ef-b74c-aa349b9c1cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:07.788670 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02ac9a-6c70-41ef-b74c-aa349b9c1cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:07.788938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02ac9a-6c70-41ef-b74c-aa349b9c1cc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:07.789569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02ac9a-6c70-41ef-b74c-aa349b9c1cc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:07.791457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02ac9a-6c70-41ef-b74c-aa349b9c1cc9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:07.819833 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1170/2337] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:25:07 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:25:07.823817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68711784-23b4-4e0d-af84-1088b209d420 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:07.833539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68711784-23b4-4e0d-af84-1088b209d420 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:07.891007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7084b4-6ec1-4333-ba69-ed01e2f4ce94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:07.891645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7084b4-6ec1-4333-ba69-ed01e2f4ce94 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:07.891924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7084b4-6ec1-4333-ba69-ed01e2f4ce94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:07.892223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7084b4-6ec1-4333-ba69-ed01e2f4ce94 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:07.892819 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7084b4-6ec1-4333-ba69-ed01e2f4ce94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:07.894716 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7084b4-6ec1-4333-ba69-ed01e2f4ce94 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:07.919348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1168/2338] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:25:07 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:25:07.923629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02ac9a-6c70-41ef-b74c-aa349b9c1cc9 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:07.934689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02ac9a-6c70-41ef-b74c-aa349b9c1cc9 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:07.939314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0604cb-3dc2-425f-93b5-80e20b0fc8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:07.940052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0604cb-3dc2-425f-93b5-80e20b0fc8ac admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:07.940332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0604cb-3dc2-425f-93b5-80e20b0fc8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:07.940799 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0604cb-3dc2-425f-93b5-80e20b0fc8ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:07.941573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0604cb-3dc2-425f-93b5-80e20b0fc8ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:07.943788 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0604cb-3dc2-425f-93b5-80e20b0fc8ac admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:07.956158 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1171/2339] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:25:07.960054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7084b4-6ec1-4333-ba69-ed01e2f4ce94 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:07.973937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7084b4-6ec1-4333-ba69-ed01e2f4ce94 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:07.978697 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd167e-ecef-497c-8244-ed9053cb6014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:07.979446 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd167e-ecef-497c-8244-ed9053cb6014 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:07.979764 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd167e-ecef-497c-8244-ed9053cb6014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:07.980036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd167e-ecef-497c-8244-ed9053cb6014 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:07.980694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd167e-ecef-497c-8244-ed9053cb6014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:07.982501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd167e-ecef-497c-8244-ed9053cb6014 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:07.995149 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1169/2340] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:25:07.999436 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0604cb-3dc2-425f-93b5-80e20b0fc8ac admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:08.014637 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0604cb-3dc2-425f-93b5-80e20b0fc8ac admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:08.030329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62636707-2e06-4ac9-b0cf-8416afd5d9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:08.031021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62636707-2e06-4ac9-b0cf-8416afd5d9f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:08.031307 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62636707-2e06-4ac9-b0cf-8416afd5d9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:08.032006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62636707-2e06-4ac9-b0cf-8416afd5d9f1 admin admin] PATH_INFO: `/v3/projects/c80feb95ecbf413ca0bc52a519bf43b7/users/dff0daad12c143fabd16d40a1939b87e/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:08.049800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd167e-ecef-497c-8244-ed9053cb6014 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:08.051711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-62bd167e-ecef-497c-8244-ed9053cb6014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:08.065644 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62636707-2e06-4ac9-b0cf-8416afd5d9f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c80feb95ecbf413ca0bc52a519bf43b7, user_id=dff0daad12c143fabd16d40a1939b87e, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:08.069531 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62636707-2e06-4ac9-b0cf-8416afd5d9f1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:08.075113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd167e-ecef-497c-8244-ed9053cb6014 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:08.083098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782a278-d941-4b23-9f8b-4c1fb305f265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:08.083784 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782a278-d941-4b23-9f8b-4c1fb305f265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:08.084292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782a278-d941-4b23-9f8b-4c1fb305f265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:08.084666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782a278-d941-4b23-9f8b-4c1fb305f265 admin admin] PATH_INFO: `/v3/users/413a44abfed9477fb8b516ddc7e44ef1` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:08.086182 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782a278-d941-4b23-9f8b-4c1fb305f265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=413a44abfed9477fb8b516ddc7e44ef1)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:08.088303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1172/2341] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:25:07 2022] PUT /identity/v3/projects/c80feb95ecbf413ca0bc52a519bf43b7/users/dff0daad12c143fabd16d40a1939b87e/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:08.092251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782a278-d941-4b23-9f8b-4c1fb305f265 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:08.093315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3d9c2-7688-4661-a271-114d7f169771 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:08.093607 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3d9c2-7688-4661-a271-114d7f169771 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:08.093886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3d9c2-7688-4661-a271-114d7f169771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:08.101922 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:08.136177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-3fe3d9c2-7688-4661-a271-114d7f169771 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:08.137700 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3fe3d9c2-7688-4661-a271-114d7f169771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:08.169839 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0782a278-d941-4b23-9f8b-4c1fb305f265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '413a44abfed9477fb8b516ddc7e44ef1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:08.181002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0782a278-d941-4b23-9f8b-4c1fb305f265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '413a44abfed9477fb8b516ddc7e44ef1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:08.181740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0782a278-d941-4b23-9f8b-4c1fb305f265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '413a44abfed9477fb8b516ddc7e44ef1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:08.182389 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0782a278-d941-4b23-9f8b-4c1fb305f265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '413a44abfed9477fb8b516ddc7e44ef1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:08.193433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0782a278-d941-4b23-9f8b-4c1fb305f265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '413a44abfed9477fb8b516ddc7e44ef1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:08.214562 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1173/2342] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:25:08.214780 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1170/2343] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:25:08 2022] DELETE /identity/v3/users/413a44abfed9477fb8b516ddc7e44ef1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:25:08.214892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7b5fd-1381-422a-832a-85e8476b41ec None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:08.214892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7b5fd-1381-422a-832a-85e8476b41ec None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:08.214892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7b5fd-1381-422a-832a-85e8476b41ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:08.217043 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d25963-c5c6-46ff-b827-bbea10a44669 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:08.217478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d25963-c5c6-46ff-b827-bbea10a44669 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:08.217828 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d25963-c5c6-46ff-b827-bbea10a44669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:08.232721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-45f7b5fd-1381-422a-832a-85e8476b41ec None None] MFA Rules not processed for user `1f4cbe342b2c4378b3c0ab2f1cd55260`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:08.233913 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-45f7b5fd-1381-422a-832a-85e8476b41ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:08.237782 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e0d25963-c5c6-46ff-b827-bbea10a44669 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:08.244401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e0d25963-c5c6-46ff-b827-bbea10a44669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:08.245335 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:08.291365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1174/2344] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:08.292317 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7b5fd-1381-422a-832a-85e8476b41ec None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:08.293563 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-45f7b5fd-1381-422a-832a-85e8476b41ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:08.315698 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1171/2345] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:08.324106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d25963-c5c6-46ff-b827-bbea10a44669 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:08.325398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e0d25963-c5c6-46ff-b827-bbea10a44669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:08.353719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7b5fd-1381-422a-832a-85e8476b41ec None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:08.358831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d392e-4051-4e7f-a8dc-a0e10b68b1d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:08.359443 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d392e-4051-4e7f-a8dc-a0e10b68b1d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:08.359764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d392e-4051-4e7f-a8dc-a0e10b68b1d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:08.360213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d392e-4051-4e7f-a8dc-a0e10b68b1d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:08.371408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d392e-4051-4e7f-a8dc-a0e10b68b1d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:08.373545 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d392e-4051-4e7f-a8dc-a0e10b68b1d7 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:08.384007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1175/2346] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:25:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:25:08.387483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d25963-c5c6-46ff-b827-bbea10a44669 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:08.391150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cef81d-4557-479c-b4a9-3c46e01f53fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:08.391804 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cef81d-4557-479c-b4a9-3c46e01f53fc service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:08.392087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cef81d-4557-479c-b4a9-3c46e01f53fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:08.392364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cef81d-4557-479c-b4a9-3c46e01f53fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:08.401726 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cef81d-4557-479c-b4a9-3c46e01f53fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:08.403161 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cef81d-4557-479c-b4a9-3c46e01f53fc service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:08.413751 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1172/2347] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:25:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:25:08.940898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d392e-4051-4e7f-a8dc-a0e10b68b1d7 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:08.943373 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8b6d392e-4051-4e7f-a8dc-a0e10b68b1d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:08.957509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d392e-4051-4e7f-a8dc-a0e10b68b1d7 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:08.962937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f30506-7433-455c-b165-3a91414fbc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:08.963662 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f30506-7433-455c-b165-3a91414fbc96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:08.963892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f30506-7433-455c-b165-3a91414fbc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:08.964159 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f30506-7433-455c-b165-3a91414fbc96 admin admin] PATH_INFO: `/v3/projects/685bb11a28794448b00df7ff778f1eff` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:08.965328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f30506-7433-455c-b165-3a91414fbc96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=685bb11a28794448b00df7ff778f1eff)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:08.967427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f30506-7433-455c-b165-3a91414fbc96 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:09.018723 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:09.019920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d1f30506-7433-455c-b165-3a91414fbc96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '685bb11a28794448b00df7ff778f1eff', 'request_id': 'req-d1f30506-7433-455c-b165-3a91414fbc96'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:09.027904 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d1f30506-7433-455c-b165-3a91414fbc96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '685bb11a28794448b00df7ff778f1eff', 'request_id': 'req-d1f30506-7433-455c-b165-3a91414fbc96'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:09.035142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d1f30506-7433-455c-b165-3a91414fbc96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '685bb11a28794448b00df7ff778f1eff', 'request_id': 'req-d1f30506-7433-455c-b165-3a91414fbc96'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:09.046740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d1f30506-7433-455c-b165-3a91414fbc96 admin admin] The token cache is being invalidate because project 685bb11a28794448b00df7ff778f1eff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:25:09.047031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d1f30506-7433-455c-b165-3a91414fbc96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:09.047719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d1f30506-7433-455c-b165-3a91414fbc96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:09.049310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1176/2348] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:25:08 2022] DELETE /identity/v3/projects/685bb11a28794448b00df7ff778f1eff => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:09.054023 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cef81d-4557-479c-b4a9-3c46e01f53fc service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:09.066229 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cef81d-4557-479c-b4a9-3c46e01f53fc service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:09.134412 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdbfed-0a1b-4b14-b77c-570005bd6601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:09.135040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdbfed-0a1b-4b14-b77c-570005bd6601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:09.135328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdbfed-0a1b-4b14-b77c-570005bd6601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:09.135636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdbfed-0a1b-4b14-b77c-570005bd6601 admin admin] PATH_INFO: `/v3/users/5d98ce2356a14a8fb0237e89e4d42842` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:09.136760 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdbfed-0a1b-4b14-b77c-570005bd6601 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d98ce2356a14a8fb0237e89e4d42842)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:09.138724 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdbfed-0a1b-4b14-b77c-570005bd6601 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:09.194991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-afcdbfed-0a1b-4b14-b77c-570005bd6601 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d98ce2356a14a8fb0237e89e4d42842'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:09.226261 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-afcdbfed-0a1b-4b14-b77c-570005bd6601 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d98ce2356a14a8fb0237e89e4d42842'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:09.227055 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-afcdbfed-0a1b-4b14-b77c-570005bd6601 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d98ce2356a14a8fb0237e89e4d42842'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:09.227719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-afcdbfed-0a1b-4b14-b77c-570005bd6601 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d98ce2356a14a8fb0237e89e4d42842'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:09.230482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:09.244945 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-afcdbfed-0a1b-4b14-b77c-570005bd6601 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d98ce2356a14a8fb0237e89e4d42842'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:09.278602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1173/2349] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:25:09 2022] DELETE /identity/v3/users/5d98ce2356a14a8fb0237e89e4d42842 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:09.897456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f30506-7433-455c-b165-3a91414fbc96 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:09.898983 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d1f30506-7433-455c-b165-3a91414fbc96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:09.918863 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f30506-7433-455c-b165-3a91414fbc96 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:10.002981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:10.003729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:10.003988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:10.004267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e admin admin] PATH_INFO: `/v3/projects/fe37a41632b74f029c60d97626d690d0` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:10.005970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe37a41632b74f029c60d97626d690d0)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:10.008158 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:10.089631 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe37a41632b74f029c60d97626d690d0', 'request_id': 'req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:10.100888 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe37a41632b74f029c60d97626d690d0', 'request_id': 'req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:10.108531 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe37a41632b74f029c60d97626d690d0', 'request_id': 'req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:10.128420 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e admin admin] The token cache is being invalidate because project fe37a41632b74f029c60d97626d690d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:25:10.129683 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:10.134471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:10.137676 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1177/2350] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:25:09 2022] DELETE /identity/v3/projects/fe37a41632b74f029c60d97626d690d0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:10.142931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdbfed-0a1b-4b14-b77c-570005bd6601 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:10.145725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-afcdbfed-0a1b-4b14-b77c-570005bd6601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:10.173516 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdbfed-0a1b-4b14-b77c-570005bd6601 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:10.263045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd847a16-34aa-4e7d-b3e6-d6d03e311b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:10.266190 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd847a16-34aa-4e7d-b3e6-d6d03e311b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:10.266510 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd847a16-34aa-4e7d-b3e6-d6d03e311b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:10.266968 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd847a16-34aa-4e7d-b3e6-d6d03e311b3e admin admin] PATH_INFO: `/v3/users/68088f88af5646c991ce8c571663713f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:10.270284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd847a16-34aa-4e7d-b3e6-d6d03e311b3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=68088f88af5646c991ce8c571663713f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:10.278989 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd847a16-34aa-4e7d-b3e6-d6d03e311b3e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:10.350096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dd847a16-34aa-4e7d-b3e6-d6d03e311b3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68088f88af5646c991ce8c571663713f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:10.361560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dd847a16-34aa-4e7d-b3e6-d6d03e311b3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68088f88af5646c991ce8c571663713f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:10.362416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dd847a16-34aa-4e7d-b3e6-d6d03e311b3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68088f88af5646c991ce8c571663713f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:10.363032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dd847a16-34aa-4e7d-b3e6-d6d03e311b3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68088f88af5646c991ce8c571663713f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:10.374347 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-dd847a16-34aa-4e7d-b3e6-d6d03e311b3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68088f88af5646c991ce8c571663713f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:10.408212 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1174/2351] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:25:10 2022] DELETE /identity/v3/users/68088f88af5646c991ce8c571663713f => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:11.100214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:11.102017 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:11.115120 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619e665-15a5-4a75-a6c4-8e5ea6b1031e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:11.207007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7db53-8dac-455c-821e-35bb0e7d1799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:11.208101 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7db53-8dac-455c-821e-35bb0e7d1799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:11.209999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7db53-8dac-455c-821e-35bb0e7d1799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:11.210995 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7db53-8dac-455c-821e-35bb0e7d1799 admin admin] PATH_INFO: `/v3/projects/8256294cc2c3490a99f41a450c3eb012` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:11.213191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7db53-8dac-455c-821e-35bb0e7d1799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8256294cc2c3490a99f41a450c3eb012)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:11.216749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7db53-8dac-455c-821e-35bb0e7d1799 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:11.231088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:11.282237 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-47a7db53-8dac-455c-821e-35bb0e7d1799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8256294cc2c3490a99f41a450c3eb012', 'request_id': 'req-47a7db53-8dac-455c-821e-35bb0e7d1799'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:11.292837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-47a7db53-8dac-455c-821e-35bb0e7d1799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8256294cc2c3490a99f41a450c3eb012', 'request_id': 'req-47a7db53-8dac-455c-821e-35bb0e7d1799'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:11.301975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-47a7db53-8dac-455c-821e-35bb0e7d1799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8256294cc2c3490a99f41a450c3eb012', 'request_id': 'req-47a7db53-8dac-455c-821e-35bb0e7d1799'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:11.314380 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-47a7db53-8dac-455c-821e-35bb0e7d1799 admin admin] The token cache is being invalidate because project 8256294cc2c3490a99f41a450c3eb012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:25:11.314672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-47a7db53-8dac-455c-821e-35bb0e7d1799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:11.315560 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-47a7db53-8dac-455c-821e-35bb0e7d1799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:11.321680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1178/2352] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:25:11 2022] DELETE /identity/v3/projects/8256294cc2c3490a99f41a450c3eb012 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:11.327621 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b56fb4-18ae-49a8-9fbe-2bc76aa563c3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:11.328561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b56fb4-18ae-49a8-9fbe-2bc76aa563c3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:11.329053 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b56fb4-18ae-49a8-9fbe-2bc76aa563c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:11.368311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f8b56fb4-18ae-49a8-9fbe-2bc76aa563c3 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:11.370156 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f8b56fb4-18ae-49a8-9fbe-2bc76aa563c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:11.453770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:11.510586 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1175/2353] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:25:11.513493 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed555fcf-d42e-4add-8376-2f80f0850feb None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:11.514243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed555fcf-d42e-4add-8376-2f80f0850feb None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:11.514994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed555fcf-d42e-4add-8376-2f80f0850feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:11.546719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ed555fcf-d42e-4add-8376-2f80f0850feb None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:11.547967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ed555fcf-d42e-4add-8376-2f80f0850feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:11.569353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1179/2354] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:11.574041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b56fb4-18ae-49a8-9fbe-2bc76aa563c3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:11.588417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b56fb4-18ae-49a8-9fbe-2bc76aa563c3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:11.595279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dbb5c-a61f-4410-8a6d-1e5ee2f7bd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:11.596301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dbb5c-a61f-4410-8a6d-1e5ee2f7bd50 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:11.596645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dbb5c-a61f-4410-8a6d-1e5ee2f7bd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:11.596989 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dbb5c-a61f-4410-8a6d-1e5ee2f7bd50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:11.597721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80dbb5c-a61f-4410-8a6d-1e5ee2f7bd50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:25:11.598098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80dbb5c-a61f-4410-8a6d-1e5ee2f7bd50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:11.600742 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80dbb5c-a61f-4410-8a6d-1e5ee2f7bd50 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:11.615603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1176/2355] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:25:11.617696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed555fcf-d42e-4add-8376-2f80f0850feb None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:11.628377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed555fcf-d42e-4add-8376-2f80f0850feb None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:11.633486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708b4d8-90f8-46e9-87d3-2cce3268b68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:11.634259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708b4d8-90f8-46e9-87d3-2cce3268b68b admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:11.634541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708b4d8-90f8-46e9-87d3-2cce3268b68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:11.634947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708b4d8-90f8-46e9-87d3-2cce3268b68b admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:11.635642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4708b4d8-90f8-46e9-87d3-2cce3268b68b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:11.637739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4708b4d8-90f8-46e9-87d3-2cce3268b68b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:11.672717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1180/2356] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:25:11 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:25:11.677124 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dbb5c-a61f-4410-8a6d-1e5ee2f7bd50 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:11.696653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dbb5c-a61f-4410-8a6d-1e5ee2f7bd50 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:11.761186 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5338142-6a08-4ff8-9161-77a680e24981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:11.761803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5338142-6a08-4ff8-9161-77a680e24981 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:11.762082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5338142-6a08-4ff8-9161-77a680e24981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:11.762354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5338142-6a08-4ff8-9161-77a680e24981 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:11.763001 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5338142-6a08-4ff8-9161-77a680e24981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:11.764987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5338142-6a08-4ff8-9161-77a680e24981 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:11.788455 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1177/2357] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:25:11 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:25:11.792869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708b4d8-90f8-46e9-87d3-2cce3268b68b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:11.806566 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708b4d8-90f8-46e9-87d3-2cce3268b68b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:11.811800 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c863dc-870c-40c1-aa38-8e823ebcf5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:11.812488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c863dc-870c-40c1-aa38-8e823ebcf5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:11.812957 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c863dc-870c-40c1-aa38-8e823ebcf5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:11.813330 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c863dc-870c-40c1-aa38-8e823ebcf5f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:11.814054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c863dc-870c-40c1-aa38-8e823ebcf5f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:11.815769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c863dc-870c-40c1-aa38-8e823ebcf5f0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:11.831563 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1181/2358] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:25:11.835027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5338142-6a08-4ff8-9161-77a680e24981 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:11.845738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5338142-6a08-4ff8-9161-77a680e24981 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:11.851753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2facb084-29d3-47fc-8f4b-1adbe3ed1836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:11.852497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facb084-29d3-47fc-8f4b-1adbe3ed1836 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:11.853290 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facb084-29d3-47fc-8f4b-1adbe3ed1836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:11.853638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facb084-29d3-47fc-8f4b-1adbe3ed1836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:11.854361 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2facb084-29d3-47fc-8f4b-1adbe3ed1836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:11.856071 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2facb084-29d3-47fc-8f4b-1adbe3ed1836 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:11.871736 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1178/2359] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:25:11.873552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c863dc-870c-40c1-aa38-8e823ebcf5f0 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:11.885408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c863dc-870c-40c1-aa38-8e823ebcf5f0 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:11.891554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fcd0e9-f986-4509-aaaf-96a9c41cd320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:11.891554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcd0e9-f986-4509-aaaf-96a9c41cd320 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:11.891554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcd0e9-f986-4509-aaaf-96a9c41cd320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:11.891915 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcd0e9-f986-4509-aaaf-96a9c41cd320 admin admin] PATH_INFO: `/v3/projects/1c822fb7252b4804aa9a0cb228b6397c/users/196f71447aaf45fd812efb9eeba2c0cb/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:11.909388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fcd0e9-f986-4509-aaaf-96a9c41cd320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c822fb7252b4804aa9a0cb228b6397c, user_id=196f71447aaf45fd812efb9eeba2c0cb, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:11.912481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fcd0e9-f986-4509-aaaf-96a9c41cd320 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:11.939326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2facb084-29d3-47fc-8f4b-1adbe3ed1836 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:11.945562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2facb084-29d3-47fc-8f4b-1adbe3ed1836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:11.960789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1182/2360] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:25:11 2022] PUT /identity/v3/projects/1c822fb7252b4804aa9a0cb228b6397c/users/196f71447aaf45fd812efb9eeba2c0cb/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:11.967636 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bb772-4ff2-420e-8551-cb2c3681c1ff None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:11.968424 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bb772-4ff2-420e-8551-cb2c3681c1ff None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:11.968855 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bb772-4ff2-420e-8551-cb2c3681c1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:12.048814 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-077bb772-4ff2-420e-8551-cb2c3681c1ff None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:12.053544 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-077bb772-4ff2-420e-8551-cb2c3681c1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:12.147225 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2facb084-29d3-47fc-8f4b-1adbe3ed1836 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:12.153237 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1183/2361] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:12.163743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bb772-4ff2-420e-8551-cb2c3681c1ff None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:12.165473 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-077bb772-4ff2-420e-8551-cb2c3681c1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:12.234554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:12.263089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:12.274121 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00958404-eb71-470d-b10c-66189e26dbbf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:12.275351 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00958404-eb71-470d-b10c-66189e26dbbf service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:12.275689 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00958404-eb71-470d-b10c-66189e26dbbf service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:12.275968 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00958404-eb71-470d-b10c-66189e26dbbf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:12.278343 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-00958404-eb71-470d-b10c-66189e26dbbf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:12.283603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bb772-4ff2-420e-8551-cb2c3681c1ff None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:12.288048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c60f50-2acc-4616-95f2-b14382104ca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:12.289335 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c60f50-2acc-4616-95f2-b14382104ca9 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:12.291561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c60f50-2acc-4616-95f2-b14382104ca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:12.292407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c60f50-2acc-4616-95f2-b14382104ca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:12.292835 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00958404-eb71-470d-b10c-66189e26dbbf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:12.294601 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00958404-eb71-470d-b10c-66189e26dbbf service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:12.306152 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c60f50-2acc-4616-95f2-b14382104ca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:12.309090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c60f50-2acc-4616-95f2-b14382104ca9 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:12.322442 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1184/2362] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:25:12.326832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1179/2363] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:25:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:25:16.829830 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daea83-c80b-4f94-b41d-0137ecdd59a0 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:16.831653 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daea83-c80b-4f94-b41d-0137ecdd59a0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:16.832221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daea83-c80b-4f94-b41d-0137ecdd59a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:16.885051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-11daea83-c80b-4f94-b41d-0137ecdd59a0 None None] MFA Rules not processed for user `8d817e129beb41168af70c3ada99aeda`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:16.887888 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-11daea83-c80b-4f94-b41d-0137ecdd59a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:16.989327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1185/2364] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:18.171408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff8790-b18f-4c64-a042-cd2fdb847cf3 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:18.171987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff8790-b18f-4c64-a042-cd2fdb847cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:18.171987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff8790-b18f-4c64-a042-cd2fdb847cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:18.205470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-23ff8790-b18f-4c64-a042-cd2fdb847cf3 None None] MFA Rules not processed for user `dff0daad12c143fabd16d40a1939b87e`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:18.207098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-23ff8790-b18f-4c64-a042-cd2fdb847cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:18.283592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1180/2365] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:18.284431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11daea83-c80b-4f94-b41d-0137ecdd59a0 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:18.286198 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-11daea83-c80b-4f94-b41d-0137ecdd59a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:18.301264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11daea83-c80b-4f94-b41d-0137ecdd59a0 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:18.306506 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183456af-ac42-4094-9e7a-63d39eeee154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:18.307122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183456af-ac42-4094-9e7a-63d39eeee154 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:18.307532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183456af-ac42-4094-9e7a-63d39eeee154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:18.307847 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183456af-ac42-4094-9e7a-63d39eeee154 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:18.308459 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183456af-ac42-4094-9e7a-63d39eeee154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:18.310678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183456af-ac42-4094-9e7a-63d39eeee154 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:18.345404 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1186/2366] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:25:18 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:25:18.347016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff8790-b18f-4c64-a042-cd2fdb847cf3 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:18.358863 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff8790-b18f-4c64-a042-cd2fdb847cf3 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:18.427641 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585caea-f0d2-45d9-b012-f07f194a23db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:18.428643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585caea-f0d2-45d9-b012-f07f194a23db admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:18.428643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585caea-f0d2-45d9-b012-f07f194a23db admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:18.428952 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585caea-f0d2-45d9-b012-f07f194a23db admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:18.429649 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585caea-f0d2-45d9-b012-f07f194a23db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:18.431889 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585caea-f0d2-45d9-b012-f07f194a23db admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:18.462988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1181/2367] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:25:18 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:25:18.465480 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183456af-ac42-4094-9e7a-63d39eeee154 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:18.478125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183456af-ac42-4094-9e7a-63d39eeee154 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:18.490942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908045f-f7d5-4826-a69b-412eaf9ba58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:18.492377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908045f-f7d5-4826-a69b-412eaf9ba58a admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:18.492666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908045f-f7d5-4826-a69b-412eaf9ba58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:18.492940 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908045f-f7d5-4826-a69b-412eaf9ba58a admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:18.493780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908045f-f7d5-4826-a69b-412eaf9ba58a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:18.496561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908045f-f7d5-4826-a69b-412eaf9ba58a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:18.516519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1187/2368] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:25:18.519929 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585caea-f0d2-45d9-b012-f07f194a23db admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:18.534674 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585caea-f0d2-45d9-b012-f07f194a23db admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:18.540049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b773c-f0ff-4735-8801-34393a910ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:18.540927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b773c-f0ff-4735-8801-34393a910ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:18.541412 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b773c-f0ff-4735-8801-34393a910ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:18.541912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b773c-f0ff-4735-8801-34393a910ac8 admin admin] PATH_INFO: `/v3/projects/7d7cefa469664b099a6f0a24e705bdad/users/a237896782e9404495e06813abb74a7b/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:18.576206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b773c-f0ff-4735-8801-34393a910ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7cefa469664b099a6f0a24e705bdad, user_id=a237896782e9404495e06813abb74a7b, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:18.582611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b773c-f0ff-4735-8801-34393a910ac8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:18.611197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908045f-f7d5-4826-a69b-412eaf9ba58a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:18.611405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1182/2369] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:25:18 2022] PUT /identity/v3/projects/7d7cefa469664b099a6f0a24e705bdad/users/a237896782e9404495e06813abb74a7b/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:18.626353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908045f-f7d5-4826-a69b-412eaf9ba58a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:18.786836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae88429-4a10-4d9d-9272-d9afef83ef1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:18.787715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae88429-4a10-4d9d-9272-d9afef83ef1c admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:18.788111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae88429-4a10-4d9d-9272-d9afef83ef1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:18.788515 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae88429-4a10-4d9d-9272-d9afef83ef1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:18.796305 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae88429-4a10-4d9d-9272-d9afef83ef1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:18.799468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae88429-4a10-4d9d-9272-d9afef83ef1c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:18.830317 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1188/2370] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:25:18.831726 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b773c-f0ff-4735-8801-34393a910ac8 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:18.846657 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b773c-f0ff-4735-8801-34393a910ac8 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:18.854066 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d9f1f-9895-47a0-bf0f-be3d4cdffffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:18.855009 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d9f1f-9895-47a0-bf0f-be3d4cdffffc admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:18.855278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d9f1f-9895-47a0-bf0f-be3d4cdffffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:18.855729 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d9f1f-9895-47a0-bf0f-be3d4cdffffc admin admin] PATH_INFO: `/v3/projects/7d7cefa469664b099a6f0a24e705bdad/users/a237896782e9404495e06813abb74a7b/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:18.858535 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d9f1f-9895-47a0-bf0f-be3d4cdffffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7cefa469664b099a6f0a24e705bdad, user_id=a237896782e9404495e06813abb74a7b, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:18.861076 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d9f1f-9895-47a0-bf0f-be3d4cdffffc admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:18.879791 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1183/2371] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:25:18 2022] PUT /identity/v3/projects/7d7cefa469664b099a6f0a24e705bdad/users/a237896782e9404495e06813abb74a7b/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:22.741552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf98de-0921-4347-a4fc-326723d8b214 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:22.742072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf98de-0921-4347-a4fc-326723d8b214 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:22.742184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf98de-0921-4347-a4fc-326723d8b214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:22.852778 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-addf98de-0921-4347-a4fc-326723d8b214 None None] MFA Rules not processed for user `196f71447aaf45fd812efb9eeba2c0cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:22.857684 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-addf98de-0921-4347-a4fc-326723d8b214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:22.986092 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:23.080340 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1189/2372] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:23.086674 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d9f1f-9895-47a0-bf0f-be3d4cdffffc admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:23.139097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:23.196090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d9f1f-9895-47a0-bf0f-be3d4cdffffc admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:23.202822 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bcf2f9-debb-4b65-b3f3-c07abe9d3506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:23.203639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcf2f9-debb-4b65-b3f3-c07abe9d3506 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:23.203925 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcf2f9-debb-4b65-b3f3-c07abe9d3506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:23.204200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcf2f9-debb-4b65-b3f3-c07abe9d3506 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:23.205003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bcf2f9-debb-4b65-b3f3-c07abe9d3506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:23.207207 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bcf2f9-debb-4b65-b3f3-c07abe9d3506 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:23.271792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1184/2373] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:25:23 2022] POST /identity/v3/projects => generated 383 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:25:23.272487 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf98de-0921-4347-a4fc-326723d8b214 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:23.292690 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf98de-0921-4347-a4fc-326723d8b214 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:23.436064 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfdde6-5879-4ff8-847b-016266781074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:23.437068 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfdde6-5879-4ff8-847b-016266781074 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:23.437504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfdde6-5879-4ff8-847b-016266781074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:23.437941 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfdde6-5879-4ff8-847b-016266781074 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:23.438748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfdde6-5879-4ff8-847b-016266781074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:23.441972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfdde6-5879-4ff8-847b-016266781074 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:23.486471 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bcf2f9-debb-4b65-b3f3-c07abe9d3506 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:23.498413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1190/2374] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:25:23 2022] POST /identity/v3/users => generated 298 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:25:23.502295 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bcf2f9-debb-4b65-b3f3-c07abe9d3506 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:23.509065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e960b99-2c93-4e9c-86e5-f908af840f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:23.511999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e960b99-2c93-4e9c-86e5-f908af840f2a admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:23.512444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e960b99-2c93-4e9c-86e5-f908af840f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:23.512887 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e960b99-2c93-4e9c-86e5-f908af840f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:23.514038 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e960b99-2c93-4e9c-86e5-f908af840f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:23.516500 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e960b99-2c93-4e9c-86e5-f908af840f2a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:23.544398 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfdde6-5879-4ff8-847b-016266781074 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:23.552273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1185/2375] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:25:23.591285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfdde6-5879-4ff8-847b-016266781074 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:23.604962 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83763a1a-cd25-4264-a05c-2e4caddf7e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:23.605762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83763a1a-cd25-4264-a05c-2e4caddf7e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:23.606126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83763a1a-cd25-4264-a05c-2e4caddf7e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:23.606411 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83763a1a-cd25-4264-a05c-2e4caddf7e3b admin admin] PATH_INFO: `/v3/projects/b82af73e810a4053a0a125077f1b22b5/users/fccbeb90078246019c9e032e500b9df5/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:23.627181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83763a1a-cd25-4264-a05c-2e4caddf7e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b82af73e810a4053a0a125077f1b22b5, user_id=fccbeb90078246019c9e032e500b9df5, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:23.632612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83763a1a-cd25-4264-a05c-2e4caddf7e3b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:23.658889 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1191/2376] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:25:23 2022] PUT /identity/v3/projects/b82af73e810a4053a0a125077f1b22b5/users/fccbeb90078246019c9e032e500b9df5/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:23.659143 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e960b99-2c93-4e9c-86e5-f908af840f2a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:23.672388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e960b99-2c93-4e9c-86e5-f908af840f2a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:23.910432 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc79ab9-83bd-4f25-b32b-2f41c59e185e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:23.912661 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc79ab9-83bd-4f25-b32b-2f41c59e185e admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:23.913676 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc79ab9-83bd-4f25-b32b-2f41c59e185e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:23.914282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc79ab9-83bd-4f25-b32b-2f41c59e185e admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:23.915300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc79ab9-83bd-4f25-b32b-2f41c59e185e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:23.917755 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc79ab9-83bd-4f25-b32b-2f41c59e185e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:23.937671 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1186/2377] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:25:23.942165 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83763a1a-cd25-4264-a05c-2e4caddf7e3b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:23.957850 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83763a1a-cd25-4264-a05c-2e4caddf7e3b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:23.976881 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c39043-a106-4f7d-9875-049a313deb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:23.980596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c39043-a106-4f7d-9875-049a313deb03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:23.985599 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c39043-a106-4f7d-9875-049a313deb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:24.005225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c39043-a106-4f7d-9875-049a313deb03 admin admin] PATH_INFO: `/v3/projects/b82af73e810a4053a0a125077f1b22b5/users/fccbeb90078246019c9e032e500b9df5/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:24.011673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c39043-a106-4f7d-9875-049a313deb03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b82af73e810a4053a0a125077f1b22b5, user_id=fccbeb90078246019c9e032e500b9df5, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:24.024689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c39043-a106-4f7d-9875-049a313deb03 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:24.066696 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1192/2378] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:25:23 2022] PUT /identity/v3/projects/b82af73e810a4053a0a125077f1b22b5/users/fccbeb90078246019c9e032e500b9df5/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:27.476551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc79ab9-83bd-4f25-b32b-2f41c59e185e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:27.478344 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-cfc79ab9-83bd-4f25-b32b-2f41c59e185e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:27.500729 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc79ab9-83bd-4f25-b32b-2f41c59e185e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:27.619434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990dcdeb-e5c9-4557-be9c-19a51eb16b20 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:27.620036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990dcdeb-e5c9-4557-be9c-19a51eb16b20 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:27.620036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990dcdeb-e5c9-4557-be9c-19a51eb16b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:27.635157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec38615-f903-4682-8b25-321b9cdf6da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:27.636005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec38615-f903-4682-8b25-321b9cdf6da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:27.636399 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec38615-f903-4682-8b25-321b9cdf6da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:27.636786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec38615-f903-4682-8b25-321b9cdf6da3 admin admin] PATH_INFO: `/v3/users/9aa8cfac6e1e43d9aed573bd77893e1e` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:27.638087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec38615-f903-4682-8b25-321b9cdf6da3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa8cfac6e1e43d9aed573bd77893e1e)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:27.640226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec38615-f903-4682-8b25-321b9cdf6da3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:27.677995 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-990dcdeb-e5c9-4557-be9c-19a51eb16b20 None None] MFA Rules not processed for user `a237896782e9404495e06813abb74a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:27.680485 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-990dcdeb-e5c9-4557-be9c-19a51eb16b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:27.746234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:27.748528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cec38615-f903-4682-8b25-321b9cdf6da3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa8cfac6e1e43d9aed573bd77893e1e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:27.759853 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cec38615-f903-4682-8b25-321b9cdf6da3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa8cfac6e1e43d9aed573bd77893e1e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:27.760704 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cec38615-f903-4682-8b25-321b9cdf6da3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa8cfac6e1e43d9aed573bd77893e1e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:27.761734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cec38615-f903-4682-8b25-321b9cdf6da3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa8cfac6e1e43d9aed573bd77893e1e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:27.778181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-cec38615-f903-4682-8b25-321b9cdf6da3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa8cfac6e1e43d9aed573bd77893e1e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:27.809924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1187/2379] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:25:27 2022] DELETE /identity/v3/users/9aa8cfac6e1e43d9aed573bd77893e1e => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:27.818217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c59133-e1b2-457c-b708-0b8f7202cd2c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:27.818731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c59133-e1b2-457c-b708-0b8f7202cd2c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:27.819171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c59133-e1b2-457c-b708-0b8f7202cd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:27.841749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1193/2380] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:27.846487 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c9c59133-e1b2-457c-b708-0b8f7202cd2c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:27.847949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c9c59133-e1b2-457c-b708-0b8f7202cd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:27.855997 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990dcdeb-e5c9-4557-be9c-19a51eb16b20 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:27.857497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-990dcdeb-e5c9-4557-be9c-19a51eb16b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:27.877676 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:28.017144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1188/2381] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:28.029645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c59133-e1b2-457c-b708-0b8f7202cd2c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:28.031632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c9c59133-e1b2-457c-b708-0b8f7202cd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:28.041083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990dcdeb-e5c9-4557-be9c-19a51eb16b20 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:28.155906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c59133-e1b2-457c-b708-0b8f7202cd2c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:28.160616 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5aa22-3013-4aab-8d4e-1fe0823f0ccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:28.161243 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5aa22-3013-4aab-8d4e-1fe0823f0ccd service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:28.161527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5aa22-3013-4aab-8d4e-1fe0823f0ccd service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:28.161803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5aa22-3013-4aab-8d4e-1fe0823f0ccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:28.175279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5aa22-3013-4aab-8d4e-1fe0823f0ccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:28.177458 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5aa22-3013-4aab-8d4e-1fe0823f0ccd service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:28.190407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235930a-cde9-48b6-b324-10bf2a778f99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:28.190667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1189/2382] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:25:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:25:28.191187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235930a-cde9-48b6-b324-10bf2a778f99 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:28.192710 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235930a-cde9-48b6-b324-10bf2a778f99 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:28.193149 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235930a-cde9-48b6-b324-10bf2a778f99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:28.197076 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6235930a-cde9-48b6-b324-10bf2a778f99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:28.331971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235930a-cde9-48b6-b324-10bf2a778f99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:28.336890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235930a-cde9-48b6-b324-10bf2a778f99 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:28.362492 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1194/2383] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:25:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:25:29.111469 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5aa22-3013-4aab-8d4e-1fe0823f0ccd service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:29.113020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-dfd5aa22-3013-4aab-8d4e-1fe0823f0ccd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:29.131705 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5aa22-3013-4aab-8d4e-1fe0823f0ccd service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:29.137135 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386f97e-62b9-4a97-ad04-1f0f56d40c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:29.137828 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386f97e-62b9-4a97-ad04-1f0f56d40c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:29.138118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386f97e-62b9-4a97-ad04-1f0f56d40c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:29.138397 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386f97e-62b9-4a97-ad04-1f0f56d40c09 admin admin] PATH_INFO: `/v3/projects/6067d5ae59724b17858fd5ab3a145bf5` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:29.139594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386f97e-62b9-4a97-ad04-1f0f56d40c09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6067d5ae59724b17858fd5ab3a145bf5)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:29.142462 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386f97e-62b9-4a97-ad04-1f0f56d40c09 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:29.260106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6386f97e-62b9-4a97-ad04-1f0f56d40c09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6067d5ae59724b17858fd5ab3a145bf5', 'request_id': 'req-6386f97e-62b9-4a97-ad04-1f0f56d40c09'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:29.275058 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6386f97e-62b9-4a97-ad04-1f0f56d40c09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6067d5ae59724b17858fd5ab3a145bf5', 'request_id': 'req-6386f97e-62b9-4a97-ad04-1f0f56d40c09'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:29.288538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6386f97e-62b9-4a97-ad04-1f0f56d40c09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6067d5ae59724b17858fd5ab3a145bf5', 'request_id': 'req-6386f97e-62b9-4a97-ad04-1f0f56d40c09'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:29.290551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:29.317971 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6386f97e-62b9-4a97-ad04-1f0f56d40c09 admin admin] The token cache is being invalidate because project 6067d5ae59724b17858fd5ab3a145bf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:25:29.318294 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6386f97e-62b9-4a97-ad04-1f0f56d40c09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:29.319214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-6386f97e-62b9-4a97-ad04-1f0f56d40c09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:25:29.320851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1190/2384] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:25:29 2022] DELETE /identity/v3/projects/6067d5ae59724b17858fd5ab3a145bf5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:29.326197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc607b-f21f-4dbb-870c-59173a1e4686 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:29.326700 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc607b-f21f-4dbb-870c-59173a1e4686 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:29.326998 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc607b-f21f-4dbb-870c-59173a1e4686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:29.356701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-82cc607b-f21f-4dbb-870c-59173a1e4686 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:29.359965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-82cc607b-f21f-4dbb-870c-59173a1e4686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:29.468605 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:29.496060 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1195/2385] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:29.504039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8899a-205d-4e43-98af-a8c9153549a7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:29.504354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8899a-205d-4e43-98af-a8c9153549a7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:29.504636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8899a-205d-4e43-98af-a8c9153549a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:29.528392 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-dea8899a-205d-4e43-98af-a8c9153549a7 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:29.529757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-dea8899a-205d-4e43-98af-a8c9153549a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:29.542451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1191/2386] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:29.544068 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc607b-f21f-4dbb-870c-59173a1e4686 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:29.555104 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc607b-f21f-4dbb-870c-59173a1e4686 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:29.560160 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ca012-3c1e-40a5-8981-f987bc64d74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:29.560805 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ca012-3c1e-40a5-8981-f987bc64d74e admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:29.561088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ca012-3c1e-40a5-8981-f987bc64d74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:29.561369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ca012-3c1e-40a5-8981-f987bc64d74e admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:29.562030 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ca012-3c1e-40a5-8981-f987bc64d74e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:25:29.562938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ca012-3c1e-40a5-8981-f987bc64d74e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:29.564158 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ca012-3c1e-40a5-8981-f987bc64d74e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:29.577689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1196/2387] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:25:29.581965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8899a-205d-4e43-98af-a8c9153549a7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:29.596025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8899a-205d-4e43-98af-a8c9153549a7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:29.605604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2dac73-78b0-458c-a40b-35885aac3404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:29.606912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dac73-78b0-458c-a40b-35885aac3404 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:29.607394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dac73-78b0-458c-a40b-35885aac3404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:29.607891 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dac73-78b0-458c-a40b-35885aac3404 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:29.608869 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2dac73-78b0-458c-a40b-35885aac3404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:29.611750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2dac73-78b0-458c-a40b-35885aac3404 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:29.671156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ca012-3c1e-40a5-8981-f987bc64d74e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:29.671584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1192/2388] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:25:29 2022] POST /identity/v3/projects => generated 381 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:25:29.691170 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ca012-3c1e-40a5-8981-f987bc64d74e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:29.847893 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c42732-8e06-465a-85eb-461a9310a142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:29.849416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c42732-8e06-465a-85eb-461a9310a142 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:29.850101 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c42732-8e06-465a-85eb-461a9310a142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:29.860414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c42732-8e06-465a-85eb-461a9310a142 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:29.863065 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c42732-8e06-465a-85eb-461a9310a142 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:29.868061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c42732-8e06-465a-85eb-461a9310a142 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:29.917051 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2dac73-78b0-458c-a40b-35885aac3404 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:29.932912 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1197/2389] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:25:29 2022] POST /identity/v3/users => generated 291 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:25:29.938332 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2dac73-78b0-458c-a40b-35885aac3404 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:29.945797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07d3c2-159d-4943-b700-cbca652846e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:29.948162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07d3c2-159d-4943-b700-cbca652846e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:29.948742 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07d3c2-159d-4943-b700-cbca652846e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:29.949202 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07d3c2-159d-4943-b700-cbca652846e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:29.950721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07d3c2-159d-4943-b700-cbca652846e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:29.952661 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07d3c2-159d-4943-b700-cbca652846e3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:29.967407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1193/2390] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:25:29.970950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c42732-8e06-465a-85eb-461a9310a142 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:29.982794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c42732-8e06-465a-85eb-461a9310a142 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:29.989369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c21e5-7850-4a12-b750-6854f0533fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:29.990357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c21e5-7850-4a12-b750-6854f0533fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:29.990735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c21e5-7850-4a12-b750-6854f0533fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:29.991156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c21e5-7850-4a12-b750-6854f0533fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:29.993626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c21e5-7850-4a12-b750-6854f0533fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:29.995621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c21e5-7850-4a12-b750-6854f0533fe1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:30.014119 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1198/2391] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:25:30.015314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07d3c2-159d-4943-b700-cbca652846e3 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:30.029007 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07d3c2-159d-4943-b700-cbca652846e3 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:30.034857 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be7b5b-7381-49d2-881a-7bc9d94a92ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:30.035764 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be7b5b-7381-49d2-881a-7bc9d94a92ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:30.036053 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be7b5b-7381-49d2-881a-7bc9d94a92ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:30.036334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be7b5b-7381-49d2-881a-7bc9d94a92ed admin admin] PATH_INFO: `/v3/projects/d2d5034a312544cd9a1d72439556cf41/users/7e650801308e4a79b259e239831491e3/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:30.057061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be7b5b-7381-49d2-881a-7bc9d94a92ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2d5034a312544cd9a1d72439556cf41, user_id=7e650801308e4a79b259e239831491e3, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:30.060160 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be7b5b-7381-49d2-881a-7bc9d94a92ed admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:30.082168 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1194/2392] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:25:30 2022] PUT /identity/v3/projects/d2d5034a312544cd9a1d72439556cf41/users/7e650801308e4a79b259e239831491e3/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:25:30.085519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a776f3-bdd4-41fb-8da9-8dbe4f3beaf0 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:30.086001 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a776f3-bdd4-41fb-8da9-8dbe4f3beaf0 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:30.086471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a776f3-bdd4-41fb-8da9-8dbe4f3beaf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:30.111277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-b5a776f3-bdd4-41fb-8da9-8dbe4f3beaf0 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:30.113009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b5a776f3-bdd4-41fb-8da9-8dbe4f3beaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:30.200801 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1199/2393] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:30.214680 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be7b5b-7381-49d2-881a-7bc9d94a92ed admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:30.214680 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f4be7b5b-7381-49d2-881a-7bc9d94a92ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:30.289718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be7b5b-7381-49d2-881a-7bc9d94a92ed admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:30.295578 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fa6ba-7a3b-4830-9f9f-b3c920b356f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:30.296389 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fa6ba-7a3b-4830-9f9f-b3c920b356f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:30.296732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fa6ba-7a3b-4830-9f9f-b3c920b356f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:30.297017 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fa6ba-7a3b-4830-9f9f-b3c920b356f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:30.308797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fa6ba-7a3b-4830-9f9f-b3c920b356f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:30.310333 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fa6ba-7a3b-4830-9f9f-b3c920b356f0 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:30.322291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1195/2394] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:25:31.318490 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ede2c2-c771-4e08-81d6-db228b59d72a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:31.319583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ede2c2-c771-4e08-81d6-db228b59d72a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:31.319996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ede2c2-c771-4e08-81d6-db228b59d72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:31.358651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a4ede2c2-c771-4e08-81d6-db228b59d72a None None] MFA Rules not processed for user `7e650801308e4a79b259e239831491e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:31.360195 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a4ede2c2-c771-4e08-81d6-db228b59d72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:31.428099 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:31.470269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1200/2395] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:31.483315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fa6ba-7a3b-4830-9f9f-b3c920b356f0 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:31.484844 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-817fa6ba-7a3b-4830-9f9f-b3c920b356f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:31.634168 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fa6ba-7a3b-4830-9f9f-b3c920b356f0 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:31.642023 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dee60-022d-4e1e-89ce-a785bb7ac0e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:31.642796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dee60-022d-4e1e-89ce-a785bb7ac0e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:31.642917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dee60-022d-4e1e-89ce-a785bb7ac0e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:31.643195 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dee60-022d-4e1e-89ce-a785bb7ac0e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:31.669585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dee60-022d-4e1e-89ce-a785bb7ac0e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:31.671490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dee60-022d-4e1e-89ce-a785bb7ac0e9 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:31.693749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1196/2396] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:25:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:25:33.327679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3d36c-3e80-40cd-82c6-15f5305f02fe None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:33.328691 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3d36c-3e80-40cd-82c6-15f5305f02fe None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:33.329179 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3d36c-3e80-40cd-82c6-15f5305f02fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:33.356681 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-f0e3d36c-3e80-40cd-82c6-15f5305f02fe None None] MFA Rules not processed for user `7e650801308e4a79b259e239831491e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:33.358105 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f0e3d36c-3e80-40cd-82c6-15f5305f02fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:33.365880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1201/2397] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:34.792739 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51929a49-9a33-4e99-9c59-07cadb032200 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:34.793227 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51929a49-9a33-4e99-9c59-07cadb032200 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:34.793320 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51929a49-9a33-4e99-9c59-07cadb032200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:34.846724 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-51929a49-9a33-4e99-9c59-07cadb032200 None None] MFA Rules not processed for user `fccbeb90078246019c9e032e500b9df5`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:34.848304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-51929a49-9a33-4e99-9c59-07cadb032200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:34.874902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:34.975798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1197/2398] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:34.990988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3d36c-3e80-40cd-82c6-15f5305f02fe None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:34.992509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f0e3d36c-3e80-40cd-82c6-15f5305f02fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:35.083950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3d36c-3e80-40cd-82c6-15f5305f02fe None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:35.087836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f1f6c-923b-45be-9770-0a7e3c196223 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a889616f86346f48b7573969174ee57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:35.088749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f1f6c-923b-45be-9770-0a7e3c196223 service glance] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:35.089258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f1f6c-923b-45be-9770-0a7e3c196223 service glance] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:35.089605 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f1f6c-923b-45be-9770-0a7e3c196223 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:35.091469 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-511f1f6c-923b-45be-9770-0a7e3c196223 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:35.179413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f1f6c-923b-45be-9770-0a7e3c196223 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:35.179413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f1f6c-923b-45be-9770-0a7e3c196223 service glance] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:35.192341 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1202/2399] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:25:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:25:35.513863 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b1b28-17b9-4f7d-96d1-bd12fd1d638a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:35.514401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b1b28-17b9-4f7d-96d1-bd12fd1d638a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:35.514920 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b1b28-17b9-4f7d-96d1-bd12fd1d638a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:35.553994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-7f5b1b28-17b9-4f7d-96d1-bd12fd1d638a None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:35.555248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7f5b1b28-17b9-4f7d-96d1-bd12fd1d638a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:35.675108 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1198/2400] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Oct 6 12:25:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:35.733557 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f1f6c-923b-45be-9770-0a7e3c196223 service glance] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:35.736714 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-511f1f6c-923b-45be-9770-0a7e3c196223 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:35.875819 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f1f6c-923b-45be-9770-0a7e3c196223 service glance] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:35.881363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e41a2-1fff-4853-9a6e-52fe91c13044 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:35.882001 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e41a2-1fff-4853-9a6e-52fe91c13044 service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:35.882276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e41a2-1fff-4853-9a6e-52fe91c13044 service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:35.882579 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e41a2-1fff-4853-9a6e-52fe91c13044 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:35.907057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e41a2-1fff-4853-9a6e-52fe91c13044 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:35.908690 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e41a2-1fff-4853-9a6e-52fe91c13044 service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:35.921882 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1203/2401] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:25:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 12:25:45.649459 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=45 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:45.916461 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=91 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:25:48.104192 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aadebc-fcfa-4bd6-83e6-a490851366ab None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:48.104720 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aadebc-fcfa-4bd6-83e6-a490851366ab None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:48.104720 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aadebc-fcfa-4bd6-83e6-a490851366ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:48.123947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-93aadebc-fcfa-4bd6-83e6-a490851366ab None None] MFA Rules not processed for user `196f71447aaf45fd812efb9eeba2c0cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:48.125162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-93aadebc-fcfa-4bd6-83e6-a490851366ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:48.132542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1199/2402] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:51.241798 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe3bc8-55ec-41bd-8e9e-728135a0379d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:51.242358 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe3bc8-55ec-41bd-8e9e-728135a0379d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:51.242358 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe3bc8-55ec-41bd-8e9e-728135a0379d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:51.283603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-4ebe3bc8-55ec-41bd-8e9e-728135a0379d None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:51.284838 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4ebe3bc8-55ec-41bd-8e9e-728135a0379d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:51.292168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1204/2403] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Oct 6 12:25:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:25:51.305254 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aadebc-fcfa-4bd6-83e6-a490851366ab None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:25:51.318122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aadebc-fcfa-4bd6-83e6-a490851366ab None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:25:51.323318 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e91d0-aa55-4e85-a35f-1bda471c4afd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:25:51.323955 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e91d0-aa55-4e85-a35f-1bda471c4afd service swift] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:51.324237 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e91d0-aa55-4e85-a35f-1bda471c4afd service swift] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:51.324509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e91d0-aa55-4e85-a35f-1bda471c4afd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:51.332847 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e91d0-aa55-4e85-a35f-1bda471c4afd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:25:51.334428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e91d0-aa55-4e85-a35f-1bda471c4afd service swift] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:25:51.342684 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1200/2404] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:25:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 12:25:58.503937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630b9df-498f-4dcc-9062-15cf48db063b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:25:58.504537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630b9df-498f-4dcc-9062-15cf48db063b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:25:58.504537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630b9df-498f-4dcc-9062-15cf48db063b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:25:58.523902 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-5630b9df-498f-4dcc-9062-15cf48db063b None None] MFA Rules not processed for user `dff0daad12c143fabd16d40a1939b87e`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:25:58.525128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5630b9df-498f-4dcc-9062-15cf48db063b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:25:58.581388 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1205/2405] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:25:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:26:01.339411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=9 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:26:08.574644 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=28 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:26:15.747012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c3c11-4d89-4187-8550-cf87053eb843 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:26:15.747625 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c3c11-4d89-4187-8550-cf87053eb843 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:26:15.747796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c3c11-4d89-4187-8550-cf87053eb843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:26:15.771247 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-570c3c11-4d89-4187-8550-cf87053eb843 None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:26:15.773101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-570c3c11-4d89-4187-8550-cf87053eb843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:26:15.786416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1201/2406] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Oct 6 12:26:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:26:15.798677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630b9df-498f-4dcc-9062-15cf48db063b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:26:15.810853 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630b9df-498f-4dcc-9062-15cf48db063b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:26:15.815944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a9a0d-9736-47b6-b8bd-2b8ddedbeede service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:26:15.816737 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a9a0d-9736-47b6-b8bd-2b8ddedbeede service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:26:15.817023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a9a0d-9736-47b6-b8bd-2b8ddedbeede service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:26:15.817353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a9a0d-9736-47b6-b8bd-2b8ddedbeede service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:26:15.826860 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a9a0d-9736-47b6-b8bd-2b8ddedbeede service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:26:15.828546 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a9a0d-9736-47b6-b8bd-2b8ddedbeede service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:26:15.837906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1206/2407] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:26:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 12:26:25.769261 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:26:25.834409 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:26:39.974224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c3c11-4d89-4187-8550-cf87053eb843 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:26:39.983325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c3c11-4d89-4187-8550-cf87053eb843 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:26:39.987024 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e20fc-dbf5-42d3-af34-de64de6fd258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:26:39.987675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e20fc-dbf5-42d3-af34-de64de6fd258 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:26:39.987962 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e20fc-dbf5-42d3-af34-de64de6fd258 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:26:39.988237 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e20fc-dbf5-42d3-af34-de64de6fd258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:26:39.990095 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d14e20fc-dbf5-42d3-af34-de64de6fd258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:26:40.054272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e20fc-dbf5-42d3-af34-de64de6fd258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:26:40.055666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e20fc-dbf5-42d3-af34-de64de6fd258 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:26:40.067734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1202/2408] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:26:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:26:41.251614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75209397-eccd-43ff-8727-1b1e95ccafb4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:26:41.252168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75209397-eccd-43ff-8727-1b1e95ccafb4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:26:41.252168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75209397-eccd-43ff-8727-1b1e95ccafb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:26:41.272336 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-75209397-eccd-43ff-8727-1b1e95ccafb4 None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:26:41.273685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-75209397-eccd-43ff-8727-1b1e95ccafb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:26:41.281289 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1207/2409] 10.222.0.33 () {58 vars in 967 bytes} [Thu Oct 6 12:26:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:26:41.294451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e20fc-dbf5-42d3-af34-de64de6fd258 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:26:41.306054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e20fc-dbf5-42d3-af34-de64de6fd258 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:26:41.311098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695aec6d-833b-4ddf-bdbb-568203e5ebce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:26:41.311755 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695aec6d-833b-4ddf-bdbb-568203e5ebce service swift] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:26:41.312040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695aec6d-833b-4ddf-bdbb-568203e5ebce service swift] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:26:41.312316 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695aec6d-833b-4ddf-bdbb-568203e5ebce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:26:41.321363 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695aec6d-833b-4ddf-bdbb-568203e5ebce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:26:41.322945 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695aec6d-833b-4ddf-bdbb-568203e5ebce service swift] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:26:41.331478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1203/2410] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:26:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 12:26:42.568891 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75209397-eccd-43ff-8727-1b1e95ccafb4 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:26:42.570442 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-75209397-eccd-43ff-8727-1b1e95ccafb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:26:42.581877 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75209397-eccd-43ff-8727-1b1e95ccafb4 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:26:42.585440 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8133e-63a5-47eb-a803-282f62e1500e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:26:42.586124 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8133e-63a5-47eb-a803-282f62e1500e service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:26:42.586353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8133e-63a5-47eb-a803-282f62e1500e service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:26:42.586706 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8133e-63a5-47eb-a803-282f62e1500e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:26:42.596935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8133e-63a5-47eb-a803-282f62e1500e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:26:42.598461 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8133e-63a5-47eb-a803-282f62e1500e service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:26:42.608398 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1208/2411] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:26:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:26:43.844447 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695aec6d-833b-4ddf-bdbb-568203e5ebce service swift] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:26:43.846063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-695aec6d-833b-4ddf-bdbb-568203e5ebce service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:26:43.862376 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695aec6d-833b-4ddf-bdbb-568203e5ebce service swift] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:26:43.867171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11a6f3-27db-49b9-8552-3fcbe055140f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:26:43.867906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11a6f3-27db-49b9-8552-3fcbe055140f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:26:43.868264 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11a6f3-27db-49b9-8552-3fcbe055140f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:26:43.868616 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11a6f3-27db-49b9-8552-3fcbe055140f admin admin] PATH_INFO: `/v3/users/8d817e129beb41168af70c3ada99aeda` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:26:43.869872 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11a6f3-27db-49b9-8552-3fcbe055140f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d817e129beb41168af70c3ada99aeda)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:26:43.871886 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11a6f3-27db-49b9-8552-3fcbe055140f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:26:43.920967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2e11a6f3-27db-49b9-8552-3fcbe055140f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d817e129beb41168af70c3ada99aeda'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:26:43.928130 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2e11a6f3-27db-49b9-8552-3fcbe055140f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d817e129beb41168af70c3ada99aeda'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:26:43.928946 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2e11a6f3-27db-49b9-8552-3fcbe055140f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d817e129beb41168af70c3ada99aeda'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:26:43.929583 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2e11a6f3-27db-49b9-8552-3fcbe055140f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d817e129beb41168af70c3ada99aeda'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:26:43.937769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2e11a6f3-27db-49b9-8552-3fcbe055140f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d817e129beb41168af70c3ada99aeda'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:26:43.957595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1204/2412] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:26:43 2022] DELETE /identity/v3/users/8d817e129beb41168af70c3ada99aeda => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:26:44.136906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8133e-63a5-47eb-a803-282f62e1500e service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:26:44.163368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8133e-63a5-47eb-a803-282f62e1500e service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:26:44.227873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82587b5e-61b1-4ed5-92f6-4c13feecd233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:26:44.228639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82587b5e-61b1-4ed5-92f6-4c13feecd233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:26:44.228996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82587b5e-61b1-4ed5-92f6-4c13feecd233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:26:44.229343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82587b5e-61b1-4ed5-92f6-4c13feecd233 admin admin] PATH_INFO: `/v3/projects/53e1ba5fc96344cebdb16d6a3e50ab8f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:26:44.230576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82587b5e-61b1-4ed5-92f6-4c13feecd233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53e1ba5fc96344cebdb16d6a3e50ab8f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:26:44.232602 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82587b5e-61b1-4ed5-92f6-4c13feecd233 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:26:44.286327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-82587b5e-61b1-4ed5-92f6-4c13feecd233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53e1ba5fc96344cebdb16d6a3e50ab8f', 'request_id': 'req-82587b5e-61b1-4ed5-92f6-4c13feecd233'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:26:44.296948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-82587b5e-61b1-4ed5-92f6-4c13feecd233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53e1ba5fc96344cebdb16d6a3e50ab8f', 'request_id': 'req-82587b5e-61b1-4ed5-92f6-4c13feecd233'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:26:44.304965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-82587b5e-61b1-4ed5-92f6-4c13feecd233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53e1ba5fc96344cebdb16d6a3e50ab8f', 'request_id': 'req-82587b5e-61b1-4ed5-92f6-4c13feecd233'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:26:44.316295 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-82587b5e-61b1-4ed5-92f6-4c13feecd233 admin admin] The token cache is being invalidate because project 53e1ba5fc96344cebdb16d6a3e50ab8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:26:44.316591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-82587b5e-61b1-4ed5-92f6-4c13feecd233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:26:44.317255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-82587b5e-61b1-4ed5-92f6-4c13feecd233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:26:44.318750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1209/2413] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:26:44 2022] DELETE /identity/v3/projects/53e1ba5fc96344cebdb16d6a3e50ab8f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:26:53.953142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=65,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:26:54.312900 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:27:21.704180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11a6f3-27db-49b9-8552-3fcbe055140f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:27:21.706134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2e11a6f3-27db-49b9-8552-3fcbe055140f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:27:21.762751 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11a6f3-27db-49b9-8552-3fcbe055140f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:27:21.810293 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d068b4-6a9e-4775-8d8e-a41624e39508 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:27:21.810895 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d068b4-6a9e-4775-8d8e-a41624e39508 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:27:21.811177 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d068b4-6a9e-4775-8d8e-a41624e39508 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:27:21.811451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d068b4-6a9e-4775-8d8e-a41624e39508 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:27:21.813192 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b9d068b4-6a9e-4775-8d8e-a41624e39508 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:27:21.880522 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d068b4-6a9e-4775-8d8e-a41624e39508 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:27:21.881932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d068b4-6a9e-4775-8d8e-a41624e39508 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:27:21.893133 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1205/2414] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:27:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:27:24.521693 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82587b5e-61b1-4ed5-92f6-4c13feecd233 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:27:24.523144 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-82587b5e-61b1-4ed5-92f6-4c13feecd233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:27:24.578847 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82587b5e-61b1-4ed5-92f6-4c13feecd233 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:27:24.583003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebeb028-1983-4ea2-9702-a8a60c59c092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:27:24.583619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebeb028-1983-4ea2-9702-a8a60c59c092 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:27:24.583902 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebeb028-1983-4ea2-9702-a8a60c59c092 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:27:24.584175 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebeb028-1983-4ea2-9702-a8a60c59c092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:27:24.585888 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bebeb028-1983-4ea2-9702-a8a60c59c092 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:27:24.599665 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebeb028-1983-4ea2-9702-a8a60c59c092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:27:24.601091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebeb028-1983-4ea2-9702-a8a60c59c092 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:27:24.612169 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1210/2415] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:27:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:27:31.887895 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=74 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:27:34.607323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=26 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:27:39.903157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd00b2-d3d4-4242-93cf-d8481f5daf8b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:27:39.903640 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd00b2-d3d4-4242-93cf-d8481f5daf8b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:27:39.903844 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd00b2-d3d4-4242-93cf-d8481f5daf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:27:39.924731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-decd00b2-d3d4-4242-93cf-d8481f5daf8b None None] MFA Rules not processed for user `7e650801308e4a79b259e239831491e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:27:39.925927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-decd00b2-d3d4-4242-93cf-d8481f5daf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:27:39.978375 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1206/2416] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:27:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:27:40.215691 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebeb028-1983-4ea2-9702-a8a60c59c092 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:27:40.224670 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebeb028-1983-4ea2-9702-a8a60c59c092 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:27:40.228223 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccdbe9-29f7-4a39-a71b-afad3229ddaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:27:40.228810 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccdbe9-29f7-4a39-a71b-afad3229ddaf service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:27:40.229091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccdbe9-29f7-4a39-a71b-afad3229ddaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:27:40.229360 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccdbe9-29f7-4a39-a71b-afad3229ddaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:27:40.231043 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c3ccdbe9-29f7-4a39-a71b-afad3229ddaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:27:40.240287 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ccdbe9-29f7-4a39-a71b-afad3229ddaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:27:40.241700 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ccdbe9-29f7-4a39-a71b-afad3229ddaf service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:27:40.249767 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1211/2417] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:27:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:27:40.552843 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd00b2-d3d4-4242-93cf-d8481f5daf8b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:27:40.554348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-decd00b2-d3d4-4242-93cf-d8481f5daf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:27:40.629284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd00b2-d3d4-4242-93cf-d8481f5daf8b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:27:40.634390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff14fd-e8d4-4a70-93e1-c6686438d7ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:27:40.634992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff14fd-e8d4-4a70-93e1-c6686438d7ec service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:27:40.635272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff14fd-e8d4-4a70-93e1-c6686438d7ec service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:27:40.635571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff14fd-e8d4-4a70-93e1-c6686438d7ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:27:40.637277 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4dff14fd-e8d4-4a70-93e1-c6686438d7ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:27:40.649012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff14fd-e8d4-4a70-93e1-c6686438d7ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:27:40.650585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff14fd-e8d4-4a70-93e1-c6686438d7ec service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:27:40.660551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1207/2418] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:27:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:27:50.246106 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:27:50.656411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=57 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:27:50.791791 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccdbe9-29f7-4a39-a71b-afad3229ddaf service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:27:50.793154 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c3ccdbe9-29f7-4a39-a71b-afad3229ddaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:27:50.804064 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccdbe9-29f7-4a39-a71b-afad3229ddaf service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:27:50.807825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b193f-9563-43ee-803d-7fc38791ca37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:27:50.808420 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b193f-9563-43ee-803d-7fc38791ca37 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:27:50.808699 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b193f-9563-43ee-803d-7fc38791ca37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:27:50.808970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b193f-9563-43ee-803d-7fc38791ca37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:27:50.810668 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b93b193f-9563-43ee-803d-7fc38791ca37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:27:50.820974 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b193f-9563-43ee-803d-7fc38791ca37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:27:50.822393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b193f-9563-43ee-803d-7fc38791ca37 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:27:50.830579 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1212/2419] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:27:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:28:00.826794 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:28:02.950770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa805d1f-262b-4bfc-8869-691924c431aa None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:02.951246 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa805d1f-262b-4bfc-8869-691924c431aa None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:02.951339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa805d1f-262b-4bfc-8869-691924c431aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:02.970879 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fa805d1f-262b-4bfc-8869-691924c431aa None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:28:02.972094 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fa805d1f-262b-4bfc-8869-691924c431aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:28:03.024914 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1208/2420] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Oct 6 12:28:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:28:03.072780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b193f-9563-43ee-803d-7fc38791ca37 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:28:03.074198 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b93b193f-9563-43ee-803d-7fc38791ca37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:28:03.145173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b193f-9563-43ee-803d-7fc38791ca37 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:28:03.150234 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f6cf6-2487-4d20-9500-c63bd9f442e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:28:03.150838 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f6cf6-2487-4d20-9500-c63bd9f442e6 service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:03.151120 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f6cf6-2487-4d20-9500-c63bd9f442e6 service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:03.151687 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f6cf6-2487-4d20-9500-c63bd9f442e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:03.160206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f6cf6-2487-4d20-9500-c63bd9f442e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:28:03.161801 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f6cf6-2487-4d20-9500-c63bd9f442e6 service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:28:03.169677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1213/2421] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:28:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 12:28:13.018485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=25 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:28:13.166433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=32 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:28:13.928494 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa805d1f-262b-4bfc-8869-691924c431aa None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:28:13.929987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fa805d1f-262b-4bfc-8869-691924c431aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:28:13.946957 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa805d1f-262b-4bfc-8869-691924c431aa None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:28:13.953072 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262c790-cc70-4725-a748-c14a3ac91eb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:28:13.953677 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262c790-cc70-4725-a748-c14a3ac91eb4 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:13.953959 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262c790-cc70-4725-a748-c14a3ac91eb4 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:13.954234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262c790-cc70-4725-a748-c14a3ac91eb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:13.955993 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f262c790-cc70-4725-a748-c14a3ac91eb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:28:14.012165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262c790-cc70-4725-a748-c14a3ac91eb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:28:14.014278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262c790-cc70-4725-a748-c14a3ac91eb4 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:28:14.023285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1209/2422] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:28:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:28:20.615490 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f6cf6-2487-4d20-9500-c63bd9f442e6 service swift] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:28:20.630090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f6cf6-2487-4d20-9500-c63bd9f442e6 service swift] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:28:20.636327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0d8da-bbba-4b79-8c89-b3d5861596c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:28:20.636937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0d8da-bbba-4b79-8c89-b3d5861596c3 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:20.637210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0d8da-bbba-4b79-8c89-b3d5861596c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:20.637480 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0d8da-bbba-4b79-8c89-b3d5861596c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:20.639205 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a7a0d8da-bbba-4b79-8c89-b3d5861596c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:28:20.656257 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0d8da-bbba-4b79-8c89-b3d5861596c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:28:20.657960 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0d8da-bbba-4b79-8c89-b3d5861596c3 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:28:20.670080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1214/2423] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:28:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:28:21.043276 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262c790-cc70-4725-a748-c14a3ac91eb4 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:28:21.052666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262c790-cc70-4725-a748-c14a3ac91eb4 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:28:21.056328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f95bc7-d66a-4b1b-a3e5-ad101c32d813 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:28:21.056937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f95bc7-d66a-4b1b-a3e5-ad101c32d813 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:21.057212 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f95bc7-d66a-4b1b-a3e5-ad101c32d813 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:21.057480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f95bc7-d66a-4b1b-a3e5-ad101c32d813 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:21.059171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e0f95bc7-d66a-4b1b-a3e5-ad101c32d813 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:28:21.068861 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f95bc7-d66a-4b1b-a3e5-ad101c32d813 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:28:21.070281 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f95bc7-d66a-4b1b-a3e5-ad101c32d813 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:28:21.079534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1210/2424] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:28:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:28:21.202800 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9218ce0-7367-4918-8ac6-db02bc02a665 None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:21.203134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9218ce0-7367-4918-8ac6-db02bc02a665 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:21.203729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9218ce0-7367-4918-8ac6-db02bc02a665 None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:21.205212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1215/2425] 10.222.0.33 () {54 vars in 921 bytes} [Thu Oct 6 12:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:28:21.211255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2755b8e-206c-478c-aa1e-cd46c9a03878 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:21.211565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2755b8e-206c-478c-aa1e-cd46c9a03878 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:21.212048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2755b8e-206c-478c-aa1e-cd46c9a03878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:21.232420 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b2755b8e-206c-478c-aa1e-cd46c9a03878 None None] MFA Rules not processed for user `5a46e63d34564d34b6b5752706375798`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:28:21.233611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b2755b8e-206c-478c-aa1e-cd46c9a03878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:28:21.242972 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1211/2426] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Oct 6 12:28:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:28:21.255034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9218ce0-7367-4918-8ac6-db02bc02a665 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:28:21.256813 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-b9218ce0-7367-4918-8ac6-db02bc02a665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:28:21.436583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9218ce0-7367-4918-8ac6-db02bc02a665 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:28:21.440103 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937113c-455e-4f2b-a297-00c89cab78fd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a889616f86346f48b7573969174ee57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:28:21.440706 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937113c-455e-4f2b-a297-00c89cab78fd service glance] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:21.440986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937113c-455e-4f2b-a297-00c89cab78fd service glance] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:21.441256 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937113c-455e-4f2b-a297-00c89cab78fd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:21.451564 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937113c-455e-4f2b-a297-00c89cab78fd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:28:21.452986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937113c-455e-4f2b-a297-00c89cab78fd service glance] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:28:21.463429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1216/2427] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:28:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:28:21.531973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5545d0-676f-4cc3-8e2c-f432a41a1914 None None] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:21.532257 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5545d0-676f-4cc3-8e2c-f432a41a1914 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:21.532571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5545d0-676f-4cc3-8e2c-f432a41a1914 None None] PATH_INFO: `/` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:21.533917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1212/2428] 10.222.0.33 () {54 vars in 921 bytes} [Thu Oct 6 12:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:28:21.538415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af9a8f-5002-4e47-bee6-c813019e15f3 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:21.538798 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af9a8f-5002-4e47-bee6-c813019e15f3 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:21.539184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af9a8f-5002-4e47-bee6-c813019e15f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:21.558382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-14af9a8f-5002-4e47-bee6-c813019e15f3 None None] MFA Rules not processed for user `2e71f9c1626f4f58916a5dd5c5987ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:28:21.559676 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-14af9a8f-5002-4e47-bee6-c813019e15f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:28:21.566101 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1217/2429] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Oct 6 12:28:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:28:21.578408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5545d0-676f-4cc3-8e2c-f432a41a1914 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:28:21.589411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5545d0-676f-4cc3-8e2c-f432a41a1914 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:28:21.593876 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4109b-392b-4b02-a3df-4836322e6f16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:28:21.594492 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4109b-392b-4b02-a3df-4836322e6f16 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:21.594770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4109b-392b-4b02-a3df-4836322e6f16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:21.595040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4109b-392b-4b02-a3df-4836322e6f16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:21.602037 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4109b-392b-4b02-a3df-4836322e6f16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:28:21.603451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4109b-392b-4b02-a3df-4836322e6f16 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:28:21.611051 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1213/2430] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:28:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:28:21.692506 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804aff7-3ee3-41a6-92ad-8257045eae7e None None] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:21.692966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804aff7-3ee3-41a6-92ad-8257045eae7e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:21.693073 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804aff7-3ee3-41a6-92ad-8257045eae7e None None] PATH_INFO: `/` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:21.694478 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1218/2431] 10.222.0.33 () {54 vars in 921 bytes} [Thu Oct 6 12:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 12:28:21.700070 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e49c31-9adb-4028-95a0-096d60d4f2ee None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:21.700354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e49c31-9adb-4028-95a0-096d60d4f2ee None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:21.700660 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e49c31-9adb-4028-95a0-096d60d4f2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:21.722062 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c0e49c31-9adb-4028-95a0-096d60d4f2ee None None] MFA Rules not processed for user `5a46e63d34564d34b6b5752706375798`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:28:21.723258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c0e49c31-9adb-4028-95a0-096d60d4f2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:28:21.732873 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1214/2432] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Oct 6 12:28:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:28:21.743038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804aff7-3ee3-41a6-92ad-8257045eae7e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:28:21.754287 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804aff7-3ee3-41a6-92ad-8257045eae7e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:28:21.757846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdbb7d-504b-4722-bf40-8936665b72c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:28:21.758428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdbb7d-504b-4722-bf40-8936665b72c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:21.758705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdbb7d-504b-4722-bf40-8936665b72c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:21.758974 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdbb7d-504b-4722-bf40-8936665b72c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:21.769318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fdbb7d-504b-4722-bf40-8936665b72c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:28:21.770733 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fdbb7d-504b-4722-bf40-8936665b72c2 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:28:21.781796 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1219/2433] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:28:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:28:27.019576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e49c31-9adb-4028-95a0-096d60d4f2ee None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:28:27.031260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e49c31-9adb-4028-95a0-096d60d4f2ee None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:28:27.036690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e747f9-a9c3-4a3a-bf35-231e3d84b629 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:28:27.037287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e747f9-a9c3-4a3a-bf35-231e3d84b629 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:27.037565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e747f9-a9c3-4a3a-bf35-231e3d84b629 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:27.037835 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e747f9-a9c3-4a3a-bf35-231e3d84b629 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:27.039584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-47e747f9-a9c3-4a3a-bf35-231e3d84b629 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:28:27.051325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e747f9-a9c3-4a3a-bf35-231e3d84b629 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:28:27.052919 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e747f9-a9c3-4a3a-bf35-231e3d84b629 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:28:27.063797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1215/2434] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:28:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:28:28.412725 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6dadd-4725-4ea5-af19-24af23362ee4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:28.413226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6dadd-4725-4ea5-af19-24af23362ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:28.413329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6dadd-4725-4ea5-af19-24af23362ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:28.432189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-31d6dadd-4725-4ea5-af19-24af23362ee4 None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:28:28.433369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-31d6dadd-4725-4ea5-af19-24af23362ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:28:28.440490 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1220/2435] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Oct 6 12:28:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:28:28.452177 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e747f9-a9c3-4a3a-bf35-231e3d84b629 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:28:28.463382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e747f9-a9c3-4a3a-bf35-231e3d84b629 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:28:28.468915 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ae3b7-d57d-4afb-b567-32e97022859a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:28:28.469617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ae3b7-d57d-4afb-b567-32e97022859a service swift] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:28.469955 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ae3b7-d57d-4afb-b567-32e97022859a service swift] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:28.470291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ae3b7-d57d-4afb-b567-32e97022859a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:28.478478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ae3b7-d57d-4afb-b567-32e97022859a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:28:28.480261 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ae3b7-d57d-4afb-b567-32e97022859a service swift] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:28:28.490003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1216/2436] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:28:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 12:28:38.427398 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=44 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:28:38.486336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=58 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:28:41.884197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6dadd-4725-4ea5-af19-24af23362ee4 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:28:41.885657 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-31d6dadd-4725-4ea5-af19-24af23362ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:28:41.901502 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6dadd-4725-4ea5-af19-24af23362ee4 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:28:41.907868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a26bba-41ad-4da6-a33a-673f0c37bc69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:28:41.908503 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a26bba-41ad-4da6-a33a-673f0c37bc69 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:41.908777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a26bba-41ad-4da6-a33a-673f0c37bc69 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:41.909049 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a26bba-41ad-4da6-a33a-673f0c37bc69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:41.910820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-89a26bba-41ad-4da6-a33a-673f0c37bc69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:28:41.970455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a26bba-41ad-4da6-a33a-673f0c37bc69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:28:41.972099 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a26bba-41ad-4da6-a33a-673f0c37bc69 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:28:41.982220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1221/2437] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:28:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:28:51.978045 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=28 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:28:56.269254 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ff36d-9599-42fc-8c3c-88c5311ea340 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:28:56.269789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ff36d-9599-42fc-8c3c-88c5311ea340 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:28:56.269789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ff36d-9599-42fc-8c3c-88c5311ea340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:28:56.288703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-333ff36d-9599-42fc-8c3c-88c5311ea340 None None] MFA Rules not processed for user `1f4cbe342b2c4378b3c0ab2f1cd55260`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:28:56.289903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-333ff36d-9599-42fc-8c3c-88c5311ea340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:28:56.302519 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1217/2438] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:28:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:29:00.224309 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a26bba-41ad-4da6-a33a-673f0c37bc69 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:29:00.225774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-89a26bba-41ad-4da6-a33a-673f0c37bc69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:29:00.242691 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a26bba-41ad-4da6-a33a-673f0c37bc69 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:29:00.246702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc507cd4-a94f-40b3-b35d-9452992ab681 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:29:00.247413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc507cd4-a94f-40b3-b35d-9452992ab681 service placement] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:29:00.248341 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc507cd4-a94f-40b3-b35d-9452992ab681 service placement] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:29:00.248620 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc507cd4-a94f-40b3-b35d-9452992ab681 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:29:00.256614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fc507cd4-a94f-40b3-b35d-9452992ab681 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:29:00.269719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc507cd4-a94f-40b3-b35d-9452992ab681 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:29:00.272238 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc507cd4-a94f-40b3-b35d-9452992ab681 service placement] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:29:00.282772 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ff36d-9599-42fc-8c3c-88c5311ea340 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:29:00.284350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-333ff36d-9599-42fc-8c3c-88c5311ea340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:29:00.285682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1222/2439] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:29:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:29:00.355490 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ff36d-9599-42fc-8c3c-88c5311ea340 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:29:00.360702 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055821f-85aa-48a1-8e70-fee2dc5fad88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:29:00.361327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055821f-85aa-48a1-8e70-fee2dc5fad88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:29:00.361609 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055821f-85aa-48a1-8e70-fee2dc5fad88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:29:00.361882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055821f-85aa-48a1-8e70-fee2dc5fad88 admin admin] PATH_INFO: `/v3/users/7e650801308e4a79b259e239831491e3` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:29:00.362997 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055821f-85aa-48a1-8e70-fee2dc5fad88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e650801308e4a79b259e239831491e3)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:29:00.365006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055821f-85aa-48a1-8e70-fee2dc5fad88 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:29:00.410116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9055821f-85aa-48a1-8e70-fee2dc5fad88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e650801308e4a79b259e239831491e3'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:29:00.416903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9055821f-85aa-48a1-8e70-fee2dc5fad88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e650801308e4a79b259e239831491e3'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:29:00.417565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9055821f-85aa-48a1-8e70-fee2dc5fad88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e650801308e4a79b259e239831491e3'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:29:00.418180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9055821f-85aa-48a1-8e70-fee2dc5fad88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e650801308e4a79b259e239831491e3'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:29:00.426351 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9055821f-85aa-48a1-8e70-fee2dc5fad88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e650801308e4a79b259e239831491e3'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:29:00.445077 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1218/2440] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:29:00 2022] DELETE /identity/v3/users/7e650801308e4a79b259e239831491e3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:29:00.616842 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc507cd4-a94f-40b3-b35d-9452992ab681 service placement] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:29:00.627621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc507cd4-a94f-40b3-b35d-9452992ab681 service placement] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:29:00.683424 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858662c2-4bd9-415f-917f-84bfcc96baef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:29:00.684076 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858662c2-4bd9-415f-917f-84bfcc96baef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:29:00.684353 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858662c2-4bd9-415f-917f-84bfcc96baef admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:29:00.684624 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858662c2-4bd9-415f-917f-84bfcc96baef admin admin] PATH_INFO: `/v3/projects/d2d5034a312544cd9a1d72439556cf41` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:29:00.685754 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858662c2-4bd9-415f-917f-84bfcc96baef admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2d5034a312544cd9a1d72439556cf41)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:29:00.687661 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858662c2-4bd9-415f-917f-84bfcc96baef admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:29:00.733796 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-858662c2-4bd9-415f-917f-84bfcc96baef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2d5034a312544cd9a1d72439556cf41', 'request_id': 'req-858662c2-4bd9-415f-917f-84bfcc96baef'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:29:00.741981 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-858662c2-4bd9-415f-917f-84bfcc96baef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2d5034a312544cd9a1d72439556cf41', 'request_id': 'req-858662c2-4bd9-415f-917f-84bfcc96baef'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:29:00.748429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-858662c2-4bd9-415f-917f-84bfcc96baef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2d5034a312544cd9a1d72439556cf41', 'request_id': 'req-858662c2-4bd9-415f-917f-84bfcc96baef'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:29:00.759131 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-858662c2-4bd9-415f-917f-84bfcc96baef admin admin] The token cache is being invalidate because project d2d5034a312544cd9a1d72439556cf41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:29:00.759416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-858662c2-4bd9-415f-917f-84bfcc96baef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:29:00.760080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-858662c2-4bd9-415f-917f-84bfcc96baef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:29:00.761552 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1223/2441] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:29:00 2022] DELETE /identity/v3/projects/d2d5034a312544cd9a1d72439556cf41 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:29:05.852514 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d788a5-d329-450b-9997-0aacd5ddaf91 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:29:05.853027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d788a5-d329-450b-9997-0aacd5ddaf91 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:29:05.853127 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d788a5-d329-450b-9997-0aacd5ddaf91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:29:05.873889 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-89d788a5-d329-450b-9997-0aacd5ddaf91 None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:29:05.875188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-89d788a5-d329-450b-9997-0aacd5ddaf91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:29:05.940002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1219/2442] 10.222.0.33 () {58 vars in 967 bytes} [Thu Oct 6 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:29:05.968842 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858662c2-4bd9-415f-917f-84bfcc96baef admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:29:05.971211 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-858662c2-4bd9-415f-917f-84bfcc96baef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:29:06.058378 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858662c2-4bd9-415f-917f-84bfcc96baef admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:29:06.064574 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d9b9ee-1c29-4d60-afc8-474c9474d5c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:29:06.067383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9b9ee-1c29-4d60-afc8-474c9474d5c5 service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:29:06.067719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9b9ee-1c29-4d60-afc8-474c9474d5c5 service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:29:06.068104 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9b9ee-1c29-4d60-afc8-474c9474d5c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:29:06.081104 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d9b9ee-1c29-4d60-afc8-474c9474d5c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:29:06.082780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d9b9ee-1c29-4d60-afc8-474c9474d5c5 service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:29:06.091196 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1224/2443] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:29:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 12:29:15.931045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:29:16.087685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:08.278717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a7ee4-0386-42d4-ad1e-9939a6106423 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:08.279281 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a7ee4-0386-42d4-ad1e-9939a6106423 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:08.279281 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a7ee4-0386-42d4-ad1e-9939a6106423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:08.310951 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-da5a7ee4-0386-42d4-ad1e-9939a6106423 None None] MFA Rules not processed for user `dff0daad12c143fabd16d40a1939b87e`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:08.312144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-da5a7ee4-0386-42d4-ad1e-9939a6106423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:08.395066 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1220/2444] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:30:14.200369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d9b9ee-1c29-4d60-afc8-474c9474d5c5 service swift] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:14.201851 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d5d9b9ee-1c29-4d60-afc8-474c9474d5c5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:14.341891 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d9b9ee-1c29-4d60-afc8-474c9474d5c5 service swift] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:14.347448 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb5171-3162-4adb-90ec-866009e2568a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:14.348094 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb5171-3162-4adb-90ec-866009e2568a service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:14.348380 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb5171-3162-4adb-90ec-866009e2568a service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:14.348677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb5171-3162-4adb-90ec-866009e2568a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:14.350417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-13cb5171-3162-4adb-90ec-866009e2568a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:14.406400 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb5171-3162-4adb-90ec-866009e2568a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:14.408010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb5171-3162-4adb-90ec-866009e2568a service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:14.416753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1225/2445] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:30:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:30:17.997150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a7ee4-0386-42d4-ad1e-9939a6106423 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:17.999414 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-da5a7ee4-0386-42d4-ad1e-9939a6106423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:18.015521 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a7ee4-0386-42d4-ad1e-9939a6106423 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:18.021354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce943ac4-95a9-4d35-a6ad-77c15e3b8560 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:18.022313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce943ac4-95a9-4d35-a6ad-77c15e3b8560 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:18.022665 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce943ac4-95a9-4d35-a6ad-77c15e3b8560 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:18.025413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce943ac4-95a9-4d35-a6ad-77c15e3b8560 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:18.026839 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ce943ac4-95a9-4d35-a6ad-77c15e3b8560 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:18.041065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce943ac4-95a9-4d35-a6ad-77c15e3b8560 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:18.042989 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce943ac4-95a9-4d35-a6ad-77c15e3b8560 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:18.057147 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1221/2446] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:30:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:30:21.113023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb5171-3162-4adb-90ec-866009e2568a service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:21.114669 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-13cb5171-3162-4adb-90ec-866009e2568a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:21.154454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:21.215372 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb5171-3162-4adb-90ec-866009e2568a service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:21.220936 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44093857-d5b9-434f-b7c1-084053167161 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:21.221565 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44093857-d5b9-434f-b7c1-084053167161 service placement] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:21.221901 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44093857-d5b9-434f-b7c1-084053167161 service placement] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:21.222173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44093857-d5b9-434f-b7c1-084053167161 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:21.223969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-44093857-d5b9-434f-b7c1-084053167161 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:21.236140 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44093857-d5b9-434f-b7c1-084053167161 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:21.237721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44093857-d5b9-434f-b7c1-084053167161 service placement] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:21.251445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1226/2447] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:30:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:30:25.787049 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0de069-bd1a-4a48-a121-d47908b5827f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:25.787587 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0de069-bd1a-4a48-a121-d47908b5827f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:25.787706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0de069-bd1a-4a48-a121-d47908b5827f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:25.807998 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-1d0de069-bd1a-4a48-a121-d47908b5827f None None] MFA Rules not processed for user `3066c1c9c79942c289b733d886d9fd0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:25.809285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1d0de069-bd1a-4a48-a121-d47908b5827f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:25.866195 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1222/2448] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:30:26.303379 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d75d6-232f-4547-9c9b-e5b2fb7af638 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:26.304310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d75d6-232f-4547-9c9b-e5b2fb7af638 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:26.304589 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d75d6-232f-4547-9c9b-e5b2fb7af638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:26.324029 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a49d75d6-232f-4547-9c9b-e5b2fb7af638 None None] MFA Rules not processed for user `ff9c115c7083441c8b363e9b685ef110`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:26.325243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a49d75d6-232f-4547-9c9b-e5b2fb7af638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:26.332445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1227/2449] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:30:29.015214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0de069-bd1a-4a48-a121-d47908b5827f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:29.028931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0de069-bd1a-4a48-a121-d47908b5827f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:29.034758 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568bdcd-9280-4a40-8f32-c914e82b2d5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:29.035370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568bdcd-9280-4a40-8f32-c914e82b2d5a service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:29.035712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568bdcd-9280-4a40-8f32-c914e82b2d5a service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:29.035967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568bdcd-9280-4a40-8f32-c914e82b2d5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:29.037664 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8568bdcd-9280-4a40-8f32-c914e82b2d5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:29.048189 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568bdcd-9280-4a40-8f32-c914e82b2d5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:29.049760 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568bdcd-9280-4a40-8f32-c914e82b2d5a service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:29.059396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1223/2450] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:30:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:30:36.318108 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=30 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:37.317426 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d75d6-232f-4547-9c9b-e5b2fb7af638 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:37.318829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a49d75d6-232f-4547-9c9b-e5b2fb7af638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:37.409764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d75d6-232f-4547-9c9b-e5b2fb7af638 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:37.415695 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8533fc8-d229-4be9-aad4-9f9faaf53c2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:37.417437 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8533fc8-d229-4be9-aad4-9f9faaf53c2d service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:37.418301 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8533fc8-d229-4be9-aad4-9f9faaf53c2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:37.418953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8533fc8-d229-4be9-aad4-9f9faaf53c2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:37.421681 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a8533fc8-d229-4be9-aad4-9f9faaf53c2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:37.540208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8533fc8-d229-4be9-aad4-9f9faaf53c2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:37.541651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8533fc8-d229-4be9-aad4-9f9faaf53c2d service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:37.563036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1228/2451] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:30:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:30:39.054599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=74 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:40.883542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568bdcd-9280-4a40-8f32-c914e82b2d5a service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:40.885191 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8568bdcd-9280-4a40-8f32-c914e82b2d5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:40.903181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568bdcd-9280-4a40-8f32-c914e82b2d5a service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:40.907931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95f7d8-98a3-43ea-abc1-94316edaab0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:40.908565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95f7d8-98a3-43ea-abc1-94316edaab0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:40.908845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95f7d8-98a3-43ea-abc1-94316edaab0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:40.909117 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95f7d8-98a3-43ea-abc1-94316edaab0a admin admin] PATH_INFO: `/v3/users/1f4cbe342b2c4378b3c0ab2f1cd55260` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:40.910277 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95f7d8-98a3-43ea-abc1-94316edaab0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f4cbe342b2c4378b3c0ab2f1cd55260)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:40.912309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95f7d8-98a3-43ea-abc1-94316edaab0a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:40.961788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ce95f7d8-98a3-43ea-abc1-94316edaab0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f4cbe342b2c4378b3c0ab2f1cd55260'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:40.969733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ce95f7d8-98a3-43ea-abc1-94316edaab0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f4cbe342b2c4378b3c0ab2f1cd55260'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:40.970765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ce95f7d8-98a3-43ea-abc1-94316edaab0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f4cbe342b2c4378b3c0ab2f1cd55260'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:40.971078 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ce95f7d8-98a3-43ea-abc1-94316edaab0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f4cbe342b2c4378b3c0ab2f1cd55260'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:40.980301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ce95f7d8-98a3-43ea-abc1-94316edaab0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f4cbe342b2c4378b3c0ab2f1cd55260'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:40.998629 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1224/2452] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:30:40 2022] DELETE /identity/v3/users/1f4cbe342b2c4378b3c0ab2f1cd55260 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:30:41.191578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8533fc8-d229-4be9-aad4-9f9faaf53c2d service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:41.202629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8533fc8-d229-4be9-aad4-9f9faaf53c2d service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:41.261557 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5dbc77-5273-4eec-8912-fe45d83a8538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:41.262193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5dbc77-5273-4eec-8912-fe45d83a8538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:41.262501 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5dbc77-5273-4eec-8912-fe45d83a8538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:41.262757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5dbc77-5273-4eec-8912-fe45d83a8538 admin admin] PATH_INFO: `/v3/projects/9138bc89f5204975a18156097530643a` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:41.263944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5dbc77-5273-4eec-8912-fe45d83a8538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9138bc89f5204975a18156097530643a)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:41.265921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5dbc77-5273-4eec-8912-fe45d83a8538 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:41.268337 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:41.332148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff5dbc77-5273-4eec-8912-fe45d83a8538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9138bc89f5204975a18156097530643a', 'request_id': 'req-ff5dbc77-5273-4eec-8912-fe45d83a8538'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:41.342541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff5dbc77-5273-4eec-8912-fe45d83a8538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9138bc89f5204975a18156097530643a', 'request_id': 'req-ff5dbc77-5273-4eec-8912-fe45d83a8538'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:41.349109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff5dbc77-5273-4eec-8912-fe45d83a8538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9138bc89f5204975a18156097530643a', 'request_id': 'req-ff5dbc77-5273-4eec-8912-fe45d83a8538'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:41.360653 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff5dbc77-5273-4eec-8912-fe45d83a8538 admin admin] The token cache is being invalidate because project 9138bc89f5204975a18156097530643a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:30:41.361079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff5dbc77-5273-4eec-8912-fe45d83a8538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:41.361740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-ff5dbc77-5273-4eec-8912-fe45d83a8538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:41.363243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1229/2453] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:30:41 2022] DELETE /identity/v3/projects/9138bc89f5204975a18156097530643a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:30:41.368570 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bceca-2110-428c-9d42-bbe8f741560e None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:41.368963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bceca-2110-428c-9d42-bbe8f741560e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:41.369354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bceca-2110-428c-9d42-bbe8f741560e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:41.441551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-841bceca-2110-428c-9d42-bbe8f741560e None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:41.442788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-841bceca-2110-428c-9d42-bbe8f741560e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:41.506594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1225/2454] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:30:41.515683 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd174f6c-b9d1-43e9-9603-e2d5be790f2e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:41.515982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd174f6c-b9d1-43e9-9603-e2d5be790f2e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:41.516250 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd174f6c-b9d1-43e9-9603-e2d5be790f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:41.536555 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-bd174f6c-b9d1-43e9-9603-e2d5be790f2e None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:41.537790 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bd174f6c-b9d1-43e9-9603-e2d5be790f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:41.546239 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1230/2455] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:30:41.550884 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bceca-2110-428c-9d42-bbe8f741560e None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:41.562815 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bceca-2110-428c-9d42-bbe8f741560e None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:41.567308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88ae88-2f43-42f5-a84e-601e0f841cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:41.567943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88ae88-2f43-42f5-a84e-601e0f841cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:41.568220 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88ae88-2f43-42f5-a84e-601e0f841cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:41.568496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88ae88-2f43-42f5-a84e-601e0f841cf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:41.569139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88ae88-2f43-42f5-a84e-601e0f841cf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:30:41.569438 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88ae88-2f43-42f5-a84e-601e0f841cf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:41.571077 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88ae88-2f43-42f5-a84e-601e0f841cf3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:41.583728 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1226/2456] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:30:41.588225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd174f6c-b9d1-43e9-9603-e2d5be790f2e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:41.597367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd174f6c-b9d1-43e9-9603-e2d5be790f2e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:41.602162 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d05f60-d6fa-4090-8564-1d4cd9383eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:41.602767 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d05f60-d6fa-4090-8564-1d4cd9383eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:41.603055 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d05f60-d6fa-4090-8564-1d4cd9383eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:41.603361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d05f60-d6fa-4090-8564-1d4cd9383eb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:41.604040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d05f60-d6fa-4090-8564-1d4cd9383eb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:41.605974 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d05f60-d6fa-4090-8564-1d4cd9383eb2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:41.635443 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1231/2457] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:30:41 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:30:41.639772 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88ae88-2f43-42f5-a84e-601e0f841cf3 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:41.649780 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88ae88-2f43-42f5-a84e-601e0f841cf3 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:41.675255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:41.707633 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aedf1a1-65a7-41ec-b601-276b49feb249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:41.708226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aedf1a1-65a7-41ec-b601-276b49feb249 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:41.708506 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aedf1a1-65a7-41ec-b601-276b49feb249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:41.708774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aedf1a1-65a7-41ec-b601-276b49feb249 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:41.709385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aedf1a1-65a7-41ec-b601-276b49feb249 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:41.711279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aedf1a1-65a7-41ec-b601-276b49feb249 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:41.732850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1227/2458] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:30:41 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:30:41.736958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d05f60-d6fa-4090-8564-1d4cd9383eb2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:41.747903 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d05f60-d6fa-4090-8564-1d4cd9383eb2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:41.752474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eaeca3-cb7d-46c7-a20f-01b2a82242ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:41.753063 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eaeca3-cb7d-46c7-a20f-01b2a82242ec admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:41.753339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eaeca3-cb7d-46c7-a20f-01b2a82242ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:41.753608 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eaeca3-cb7d-46c7-a20f-01b2a82242ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:41.754239 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eaeca3-cb7d-46c7-a20f-01b2a82242ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:41.755861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eaeca3-cb7d-46c7-a20f-01b2a82242ec admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:41.765971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1232/2459] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:30:41.769867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aedf1a1-65a7-41ec-b601-276b49feb249 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:41.780012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aedf1a1-65a7-41ec-b601-276b49feb249 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:41.784694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cab42-3c89-4cc5-8fae-b0126e386dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:41.785294 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cab42-3c89-4cc5-8fae-b0126e386dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:41.785578 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cab42-3c89-4cc5-8fae-b0126e386dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:41.785851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cab42-3c89-4cc5-8fae-b0126e386dd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:41.786488 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cab42-3c89-4cc5-8fae-b0126e386dd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:41.788101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cab42-3c89-4cc5-8fae-b0126e386dd2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:41.798766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1228/2460] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:30:41.803411 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eaeca3-cb7d-46c7-a20f-01b2a82242ec admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:41.814531 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eaeca3-cb7d-46c7-a20f-01b2a82242ec admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:41.820033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a0aa5-98a9-4b42-9975-ed84298a12d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:41.820682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a0aa5-98a9-4b42-9975-ed84298a12d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:41.820958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a0aa5-98a9-4b42-9975-ed84298a12d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:41.821229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a0aa5-98a9-4b42-9975-ed84298a12d9 admin admin] PATH_INFO: `/v3/projects/8fc55bc6e41843008da265a1e09e69e0/users/1670b0971f584c629e0c5fa018f57156/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:41.838202 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a0aa5-98a9-4b42-9975-ed84298a12d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fc55bc6e41843008da265a1e09e69e0, user_id=1670b0971f584c629e0c5fa018f57156, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:41.840953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a0aa5-98a9-4b42-9975-ed84298a12d9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:41.855820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1233/2461] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:30:41 2022] PUT /identity/v3/projects/8fc55bc6e41843008da265a1e09e69e0/users/1670b0971f584c629e0c5fa018f57156/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:30:41.865776 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e214907-c85e-4407-84b9-5c52f7d3a36d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:41.866065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e214907-c85e-4407-84b9-5c52f7d3a36d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:41.866341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e214907-c85e-4407-84b9-5c52f7d3a36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:41.894474 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-7e214907-c85e-4407-84b9-5c52f7d3a36d None None] MFA Rules not processed for user `1670b0971f584c629e0c5fa018f57156`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:41.895711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7e214907-c85e-4407-84b9-5c52f7d3a36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:41.964632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1229/2462] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:30:41.975817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a0aa5-98a9-4b42-9975-ed84298a12d9 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:41.977610 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1c9a0aa5-98a9-4b42-9975-ed84298a12d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:42.073751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a0aa5-98a9-4b42-9975-ed84298a12d9 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:42.079765 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce8131-42ec-4d9e-acb9-7e77f4fc0a64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:42.080495 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce8131-42ec-4d9e-acb9-7e77f4fc0a64 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:42.080914 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce8131-42ec-4d9e-acb9-7e77f4fc0a64 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:42.081158 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce8131-42ec-4d9e-acb9-7e77f4fc0a64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:42.092256 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce8131-42ec-4d9e-acb9-7e77f4fc0a64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:42.093901 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce8131-42ec-4d9e-acb9-7e77f4fc0a64 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:42.103955 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1234/2463] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:30:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:30:42.738176 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e214907-c85e-4407-84b9-5c52f7d3a36d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:42.751625 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e214907-c85e-4407-84b9-5c52f7d3a36d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:42.757025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df067e1e-1558-42ae-97de-1e9ce00cdc3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:42.757744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df067e1e-1558-42ae-97de-1e9ce00cdc3b service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:42.758028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df067e1e-1558-42ae-97de-1e9ce00cdc3b service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:42.758306 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df067e1e-1558-42ae-97de-1e9ce00cdc3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:42.760074 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-df067e1e-1558-42ae-97de-1e9ce00cdc3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:42.820460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df067e1e-1558-42ae-97de-1e9ce00cdc3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:42.822067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df067e1e-1558-42ae-97de-1e9ce00cdc3b service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:42.831402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1230/2464] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:30:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:30:51.702822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce8131-42ec-4d9e-acb9-7e77f4fc0a64 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:51.728761 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:51.798820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce8131-42ec-4d9e-acb9-7e77f4fc0a64 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:51.808946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad18d45-9f22-4f13-a2f8-a16fd2f9c8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:51.810115 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad18d45-9f22-4f13-a2f8-a16fd2f9c8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:51.810409 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad18d45-9f22-4f13-a2f8-a16fd2f9c8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:51.810688 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad18d45-9f22-4f13-a2f8-a16fd2f9c8ff admin admin] PATH_INFO: `/v3/users/1670b0971f584c629e0c5fa018f57156` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:51.811943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad18d45-9f22-4f13-a2f8-a16fd2f9c8ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=1670b0971f584c629e0c5fa018f57156)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:51.814425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad18d45-9f22-4f13-a2f8-a16fd2f9c8ff admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:51.886938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3ad18d45-9f22-4f13-a2f8-a16fd2f9c8ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1670b0971f584c629e0c5fa018f57156'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:51.896989 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3ad18d45-9f22-4f13-a2f8-a16fd2f9c8ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1670b0971f584c629e0c5fa018f57156'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:51.897769 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3ad18d45-9f22-4f13-a2f8-a16fd2f9c8ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1670b0971f584c629e0c5fa018f57156'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:51.898375 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3ad18d45-9f22-4f13-a2f8-a16fd2f9c8ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1670b0971f584c629e0c5fa018f57156'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:51.913386 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3ad18d45-9f22-4f13-a2f8-a16fd2f9c8ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1670b0971f584c629e0c5fa018f57156'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:51.942749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1235/2465] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:30:51 2022] DELETE /identity/v3/users/1670b0971f584c629e0c5fa018f57156 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:30:51.947130 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14fa0b-a0ca-4810-9a8b-67ecc0ccd5e6 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:51.948060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14fa0b-a0ca-4810-9a8b-67ecc0ccd5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:51.948507 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14fa0b-a0ca-4810-9a8b-67ecc0ccd5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:51.978817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-7c14fa0b-a0ca-4810-9a8b-67ecc0ccd5e6 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:51.980253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7c14fa0b-a0ca-4810-9a8b-67ecc0ccd5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:52.028396 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:52.076134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1231/2466] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:30:52.088046 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad18d45-9f22-4f13-a2f8-a16fd2f9c8ff admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:52.089950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3ad18d45-9f22-4f13-a2f8-a16fd2f9c8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:52.170185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad18d45-9f22-4f13-a2f8-a16fd2f9c8ff admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:52.174307 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7a81f-73a8-4c29-9ef1-f536c0c15c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:52.174939 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7a81f-73a8-4c29-9ef1-f536c0c15c4e service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:52.175216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7a81f-73a8-4c29-9ef1-f536c0c15c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:52.175481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7a81f-73a8-4c29-9ef1-f536c0c15c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:52.186478 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7a81f-73a8-4c29-9ef1-f536c0c15c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:52.188179 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7a81f-73a8-4c29-9ef1-f536c0c15c4e service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:52.202855 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1236/2467] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:30:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:30:52.528581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14fa0b-a0ca-4810-9a8b-67ecc0ccd5e6 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:52.530164 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7c14fa0b-a0ca-4810-9a8b-67ecc0ccd5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:52.546183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14fa0b-a0ca-4810-9a8b-67ecc0ccd5e6 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:52.553248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a56c4-17f5-4616-b29b-e22449bc73a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:52.553950 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a56c4-17f5-4616-b29b-e22449bc73a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:52.554329 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a56c4-17f5-4616-b29b-e22449bc73a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:52.554613 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a56c4-17f5-4616-b29b-e22449bc73a4 admin admin] PATH_INFO: `/v3/projects/8fc55bc6e41843008da265a1e09e69e0` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:52.557487 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a56c4-17f5-4616-b29b-e22449bc73a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fc55bc6e41843008da265a1e09e69e0)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:52.559561 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a56c4-17f5-4616-b29b-e22449bc73a4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:52.617116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2a2a56c4-17f5-4616-b29b-e22449bc73a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fc55bc6e41843008da265a1e09e69e0', 'request_id': 'req-2a2a56c4-17f5-4616-b29b-e22449bc73a4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:52.627332 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2a2a56c4-17f5-4616-b29b-e22449bc73a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fc55bc6e41843008da265a1e09e69e0', 'request_id': 'req-2a2a56c4-17f5-4616-b29b-e22449bc73a4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:52.634888 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2a2a56c4-17f5-4616-b29b-e22449bc73a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fc55bc6e41843008da265a1e09e69e0', 'request_id': 'req-2a2a56c4-17f5-4616-b29b-e22449bc73a4'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:52.647426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2a2a56c4-17f5-4616-b29b-e22449bc73a4 admin admin] The token cache is being invalidate because project 8fc55bc6e41843008da265a1e09e69e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:30:52.647800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2a2a56c4-17f5-4616-b29b-e22449bc73a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:52.648471 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2a2a56c4-17f5-4616-b29b-e22449bc73a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:52.652679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1232/2468] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:30:52 2022] DELETE /identity/v3/projects/8fc55bc6e41843008da265a1e09e69e0 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:30:52.655254 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21cecc-0059-472f-af03-ba8ccb65c802 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:52.655623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21cecc-0059-472f-af03-ba8ccb65c802 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:52.655905 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21cecc-0059-472f-af03-ba8ccb65c802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:52.678847 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-cf21cecc-0059-472f-af03-ba8ccb65c802 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:52.681778 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-cf21cecc-0059-472f-af03-ba8ccb65c802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:52.724218 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:52.764790 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1237/2469] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:30:52.772768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cb3eb-db38-4695-931b-4d9e559c0150 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:52.773329 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cb3eb-db38-4695-931b-4d9e559c0150 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:52.773962 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cb3eb-db38-4695-931b-4d9e559c0150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:52.798728 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-988cb3eb-db38-4695-931b-4d9e559c0150 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:52.799992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-988cb3eb-db38-4695-931b-4d9e559c0150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:52.813337 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1233/2470] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:30:52.814785 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21cecc-0059-472f-af03-ba8ccb65c802 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:52.825313 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21cecc-0059-472f-af03-ba8ccb65c802 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:52.837973 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd597b9f-675b-40f2-b259-4fc8c87df425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:52.838610 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd597b9f-675b-40f2-b259-4fc8c87df425 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:52.838908 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd597b9f-675b-40f2-b259-4fc8c87df425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:52.839156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd597b9f-675b-40f2-b259-4fc8c87df425 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:52.839861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd597b9f-675b-40f2-b259-4fc8c87df425 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:30:52.840164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd597b9f-675b-40f2-b259-4fc8c87df425 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:52.841760 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd597b9f-675b-40f2-b259-4fc8c87df425 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:52.857238 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1238/2471] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:30:52.861645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cb3eb-db38-4695-931b-4d9e559c0150 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:52.872237 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cb3eb-db38-4695-931b-4d9e559c0150 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:52.878694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d91e0f-8971-443e-8a42-0832a0a6566a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:52.879889 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d91e0f-8971-443e-8a42-0832a0a6566a admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:52.880265 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d91e0f-8971-443e-8a42-0832a0a6566a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:52.880606 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d91e0f-8971-443e-8a42-0832a0a6566a admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:52.881390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d91e0f-8971-443e-8a42-0832a0a6566a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:52.883304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d91e0f-8971-443e-8a42-0832a0a6566a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:52.923311 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1234/2472] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:30:52 2022] POST /identity/v3/projects => generated 367 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:30:52.925318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd597b9f-675b-40f2-b259-4fc8c87df425 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:52.943118 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd597b9f-675b-40f2-b259-4fc8c87df425 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:53.031062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c08c30-56fd-4af8-8387-fed8a1ffa795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:53.032643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c08c30-56fd-4af8-8387-fed8a1ffa795 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:53.032643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c08c30-56fd-4af8-8387-fed8a1ffa795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:53.032643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c08c30-56fd-4af8-8387-fed8a1ffa795 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:53.033011 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c08c30-56fd-4af8-8387-fed8a1ffa795 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:53.035220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c08c30-56fd-4af8-8387-fed8a1ffa795 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:53.071030 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1239/2473] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:30:52 2022] POST /identity/v3/users => generated 284 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:30:53.075625 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d91e0f-8971-443e-8a42-0832a0a6566a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:53.095776 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d91e0f-8971-443e-8a42-0832a0a6566a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:53.108989 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c48ae4-058a-491d-829b-a61ee5a15ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:53.109969 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c48ae4-058a-491d-829b-a61ee5a15ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:53.110249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c48ae4-058a-491d-829b-a61ee5a15ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:53.110520 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c48ae4-058a-491d-829b-a61ee5a15ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:53.111170 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c48ae4-058a-491d-829b-a61ee5a15ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:53.114662 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c48ae4-058a-491d-829b-a61ee5a15ea5 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:53.126718 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1235/2474] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:30:53.131071 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c08c30-56fd-4af8-8387-fed8a1ffa795 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:53.142128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c08c30-56fd-4af8-8387-fed8a1ffa795 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:53.152072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234d753-29f1-4a07-9108-076e45066f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:53.152691 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234d753-29f1-4a07-9108-076e45066f8b admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:53.153053 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234d753-29f1-4a07-9108-076e45066f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:53.153226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234d753-29f1-4a07-9108-076e45066f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:53.153899 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234d753-29f1-4a07-9108-076e45066f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:53.155814 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234d753-29f1-4a07-9108-076e45066f8b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:53.171139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1240/2475] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:30:53.172364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c48ae4-058a-491d-829b-a61ee5a15ea5 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:53.187108 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c48ae4-058a-491d-829b-a61ee5a15ea5 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:53.197211 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7564ee-4ed1-4d15-bf69-b1e791deabf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:53.199005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7564ee-4ed1-4d15-bf69-b1e791deabf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:53.199439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7564ee-4ed1-4d15-bf69-b1e791deabf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:53.200127 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7564ee-4ed1-4d15-bf69-b1e791deabf7 admin admin] PATH_INFO: `/v3/projects/225a499e61324abfbd1859ee840a23c3/users/48288b032b4a4f9c94830f2e5c89f97c/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:53.223182 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7564ee-4ed1-4d15-bf69-b1e791deabf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=225a499e61324abfbd1859ee840a23c3, user_id=48288b032b4a4f9c94830f2e5c89f97c, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:53.226176 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7564ee-4ed1-4d15-bf69-b1e791deabf7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:53.243934 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1236/2476] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:30:53 2022] PUT /identity/v3/projects/225a499e61324abfbd1859ee840a23c3/users/48288b032b4a4f9c94830f2e5c89f97c/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:30:53.255889 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd0351-f13d-436f-95af-6697477d03cd None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:53.256174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd0351-f13d-436f-95af-6697477d03cd None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:53.256453 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd0351-f13d-436f-95af-6697477d03cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:53.307147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-fdbd0351-f13d-436f-95af-6697477d03cd None None] MFA Rules not processed for user `48288b032b4a4f9c94830f2e5c89f97c`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:53.308833 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fdbd0351-f13d-436f-95af-6697477d03cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:53.441561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1241/2477] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:30:53.442725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7564ee-4ed1-4d15-bf69-b1e791deabf7 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:53.532630 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:53.535960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7564ee-4ed1-4d15-bf69-b1e791deabf7 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:53.551952 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec0028-a38c-40b2-9b44-04caa8a3f82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:53.552654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec0028-a38c-40b2-9b44-04caa8a3f82e admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:53.553015 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec0028-a38c-40b2-9b44-04caa8a3f82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:53.553348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec0028-a38c-40b2-9b44-04caa8a3f82e admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:53.554969 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec0028-a38c-40b2-9b44-04caa8a3f82e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:53.557100 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec0028-a38c-40b2-9b44-04caa8a3f82e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:53.603309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1237/2478] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:30:53 2022] POST /identity/v3/projects => generated 369 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:30:53.607265 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd0351-f13d-436f-95af-6697477d03cd None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:53.619670 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd0351-f13d-436f-95af-6697477d03cd None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:53.660511 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:53.695763 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8b469-70d4-441b-8a3e-890a8d44b034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:53.697243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8b469-70d4-441b-8a3e-890a8d44b034 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:53.697603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8b469-70d4-441b-8a3e-890a8d44b034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:53.697952 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8b469-70d4-441b-8a3e-890a8d44b034 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:53.698700 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e8b469-70d4-441b-8a3e-890a8d44b034 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:53.701043 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e8b469-70d4-441b-8a3e-890a8d44b034 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:53.726918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1242/2479] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:30:53 2022] POST /identity/v3/users => generated 291 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:30:53.731018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec0028-a38c-40b2-9b44-04caa8a3f82e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:53.746193 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec0028-a38c-40b2-9b44-04caa8a3f82e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:53.752298 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f866da5-dcfe-4044-9e08-0e052f160fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:53.752924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f866da5-dcfe-4044-9e08-0e052f160fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:53.753205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f866da5-dcfe-4044-9e08-0e052f160fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:53.753479 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f866da5-dcfe-4044-9e08-0e052f160fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:53.754127 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f866da5-dcfe-4044-9e08-0e052f160fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:53.755771 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f866da5-dcfe-4044-9e08-0e052f160fd8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:53.769420 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1238/2480] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:30:53.772613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8b469-70d4-441b-8a3e-890a8d44b034 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:53.786439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8b469-70d4-441b-8a3e-890a8d44b034 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:53.792222 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f5367-0ef6-48a3-80d3-4a1b468aef09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:53.793041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f5367-0ef6-48a3-80d3-4a1b468aef09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:53.793213 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f5367-0ef6-48a3-80d3-4a1b468aef09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:53.793429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f5367-0ef6-48a3-80d3-4a1b468aef09 admin admin] PATH_INFO: `/v3/projects/58b0fecf98bc490c9cb6937e9bb917bd/users/ea8f562a1f24439c8732285d2fa1588e/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:53.811367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f5367-0ef6-48a3-80d3-4a1b468aef09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58b0fecf98bc490c9cb6937e9bb917bd, user_id=ea8f562a1f24439c8732285d2fa1588e, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:53.814058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f5367-0ef6-48a3-80d3-4a1b468aef09 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:53.829536 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1243/2481] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:30:53 2022] PUT /identity/v3/projects/58b0fecf98bc490c9cb6937e9bb917bd/users/ea8f562a1f24439c8732285d2fa1588e/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:30:53.833623 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f866da5-dcfe-4044-9e08-0e052f160fd8 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:53.848462 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f866da5-dcfe-4044-9e08-0e052f160fd8 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:53.919788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c2cf2-d1e3-426b-9506-698bd5fb1345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:53.920467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c2cf2-d1e3-426b-9506-698bd5fb1345 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:53.920684 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c2cf2-d1e3-426b-9506-698bd5fb1345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:53.921054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c2cf2-d1e3-426b-9506-698bd5fb1345 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:53.921641 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c2cf2-d1e3-426b-9506-698bd5fb1345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:53.923195 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c2cf2-d1e3-426b-9506-698bd5fb1345 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:53.934779 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1239/2482] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:30:53.938779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f5367-0ef6-48a3-80d3-4a1b468aef09 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:53.955071 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f5367-0ef6-48a3-80d3-4a1b468aef09 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:53.959868 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01e153-356e-49a4-ba4f-38e181040693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:53.960535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01e153-356e-49a4-ba4f-38e181040693 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:53.960837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01e153-356e-49a4-ba4f-38e181040693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:53.961280 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01e153-356e-49a4-ba4f-38e181040693 admin admin] PATH_INFO: `/v3/projects/58b0fecf98bc490c9cb6937e9bb917bd/users/ea8f562a1f24439c8732285d2fa1588e/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:53.964001 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01e153-356e-49a4-ba4f-38e181040693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58b0fecf98bc490c9cb6937e9bb917bd, user_id=ea8f562a1f24439c8732285d2fa1588e, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:53.967123 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01e153-356e-49a4-ba4f-38e181040693 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:53.985490 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1244/2483] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:30:53 2022] PUT /identity/v3/projects/58b0fecf98bc490c9cb6937e9bb917bd/users/ea8f562a1f24439c8732285d2fa1588e/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:30:53.989616 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6ec9b-e816-415f-a80c-14bf5a5e2f1f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:53.990006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6ec9b-e816-415f-a80c-14bf5a5e2f1f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:53.990463 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6ec9b-e816-415f-a80c-14bf5a5e2f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:54.020943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d8c6ec9b-e816-415f-a80c-14bf5a5e2f1f None None] MFA Rules not processed for user `ea8f562a1f24439c8732285d2fa1588e`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:54.022406 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d8c6ec9b-e816-415f-a80c-14bf5a5e2f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:54.105314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1240/2484] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:30:54.118642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01e153-356e-49a4-ba4f-38e181040693 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:54.120460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fb01e153-356e-49a4-ba4f-38e181040693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:54.209774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01e153-356e-49a4-ba4f-38e181040693 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:54.214221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b014c-a983-475c-8e30-8db150836a29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:54.214890 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b014c-a983-475c-8e30-8db150836a29 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:54.215354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b014c-a983-475c-8e30-8db150836a29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:54.215704 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b014c-a983-475c-8e30-8db150836a29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:54.225497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b014c-a983-475c-8e30-8db150836a29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:54.226935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b014c-a983-475c-8e30-8db150836a29 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:54.239191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1245/2485] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:30:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:30:55.730433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6ec9b-e816-415f-a80c-14bf5a5e2f1f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:55.783537 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:55.796301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6ec9b-e816-415f-a80c-14bf5a5e2f1f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:55.801012 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c1027-9783-4bef-997d-ad924d203c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:55.801647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c1027-9783-4bef-997d-ad924d203c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:55.801959 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c1027-9783-4bef-997d-ad924d203c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:55.802236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c1027-9783-4bef-997d-ad924d203c19 admin admin] PATH_INFO: `/v3/users/48288b032b4a4f9c94830f2e5c89f97c` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:55.803410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c1027-9783-4bef-997d-ad924d203c19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48288b032b4a4f9c94830f2e5c89f97c)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:55.805508 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c1027-9783-4bef-997d-ad924d203c19 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:55.862828 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2e7c1027-9783-4bef-997d-ad924d203c19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48288b032b4a4f9c94830f2e5c89f97c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:55.873960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2e7c1027-9783-4bef-997d-ad924d203c19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48288b032b4a4f9c94830f2e5c89f97c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:55.874810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2e7c1027-9783-4bef-997d-ad924d203c19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48288b032b4a4f9c94830f2e5c89f97c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:55.875659 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2e7c1027-9783-4bef-997d-ad924d203c19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48288b032b4a4f9c94830f2e5c89f97c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:55.886549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-2e7c1027-9783-4bef-997d-ad924d203c19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48288b032b4a4f9c94830f2e5c89f97c'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:55.907241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1241/2486] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:30:55 2022] DELETE /identity/v3/users/48288b032b4a4f9c94830f2e5c89f97c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:30:55.911876 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e72cca-3d2d-43b1-8f8e-d746c4540a1e None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:55.912355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e72cca-3d2d-43b1-8f8e-d746c4540a1e None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:55.912508 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e72cca-3d2d-43b1-8f8e-d746c4540a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:55.937730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a8e72cca-3d2d-43b1-8f8e-d746c4540a1e None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:55.938960 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a8e72cca-3d2d-43b1-8f8e-d746c4540a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:56.011307 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1246/2487] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:30:56.024630 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c1027-9783-4bef-997d-ad924d203c19 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:56.026555 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2e7c1027-9783-4bef-997d-ad924d203c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:56.104431 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c1027-9783-4bef-997d-ad924d203c19 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:56.108787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d559c-56b9-4b45-b82b-05a8bd4afe4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:56.109403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d559c-56b9-4b45-b82b-05a8bd4afe4b service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:56.109686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d559c-56b9-4b45-b82b-05a8bd4afe4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:56.109966 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d559c-56b9-4b45-b82b-05a8bd4afe4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:56.119779 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d559c-56b9-4b45-b82b-05a8bd4afe4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:56.121477 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d559c-56b9-4b45-b82b-05a8bd4afe4b service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:56.131940 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1242/2488] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:30:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:30:56.548095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e72cca-3d2d-43b1-8f8e-d746c4540a1e None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:56.549891 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a8e72cca-3d2d-43b1-8f8e-d746c4540a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:56.567987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e72cca-3d2d-43b1-8f8e-d746c4540a1e None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:56.591351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b39756-b3ce-418b-89e9-293310f4e95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:56.592134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b39756-b3ce-418b-89e9-293310f4e95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:56.592464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b39756-b3ce-418b-89e9-293310f4e95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:56.592851 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b39756-b3ce-418b-89e9-293310f4e95d admin admin] PATH_INFO: `/v3/projects/225a499e61324abfbd1859ee840a23c3` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:56.594352 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b39756-b3ce-418b-89e9-293310f4e95d admin admin] RBAC: Authorizing `identity:delete_project(project_id=225a499e61324abfbd1859ee840a23c3)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:56.596188 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b39756-b3ce-418b-89e9-293310f4e95d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:56.598144 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:56.660639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-72b39756-b3ce-418b-89e9-293310f4e95d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '225a499e61324abfbd1859ee840a23c3', 'request_id': 'req-72b39756-b3ce-418b-89e9-293310f4e95d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:56.668163 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-72b39756-b3ce-418b-89e9-293310f4e95d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '225a499e61324abfbd1859ee840a23c3', 'request_id': 'req-72b39756-b3ce-418b-89e9-293310f4e95d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:56.674624 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-72b39756-b3ce-418b-89e9-293310f4e95d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '225a499e61324abfbd1859ee840a23c3', 'request_id': 'req-72b39756-b3ce-418b-89e9-293310f4e95d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:56.685455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-72b39756-b3ce-418b-89e9-293310f4e95d admin admin] The token cache is being invalidate because project 225a499e61324abfbd1859ee840a23c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:30:56.685747 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-72b39756-b3ce-418b-89e9-293310f4e95d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:56.686405 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-72b39756-b3ce-418b-89e9-293310f4e95d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:56.688151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1247/2489] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:30:56 2022] DELETE /identity/v3/projects/225a499e61324abfbd1859ee840a23c3 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:30:56.692059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d559c-56b9-4b45-b82b-05a8bd4afe4b service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:56.702794 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d559c-56b9-4b45-b82b-05a8bd4afe4b service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:56.767765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c48ca3-c482-4071-a5ea-366e698a7f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:56.768479 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c48ca3-c482-4071-a5ea-366e698a7f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:56.768740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c48ca3-c482-4071-a5ea-366e698a7f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:56.769176 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c48ca3-c482-4071-a5ea-366e698a7f56 admin admin] PATH_INFO: `/v3/users/ea8f562a1f24439c8732285d2fa1588e` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:56.770657 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c48ca3-c482-4071-a5ea-366e698a7f56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea8f562a1f24439c8732285d2fa1588e)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:56.773653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c48ca3-c482-4071-a5ea-366e698a7f56 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:56.793074 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:56.833201 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-66c48ca3-c482-4071-a5ea-366e698a7f56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea8f562a1f24439c8732285d2fa1588e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:56.845194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-66c48ca3-c482-4071-a5ea-366e698a7f56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea8f562a1f24439c8732285d2fa1588e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:56.846705 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-66c48ca3-c482-4071-a5ea-366e698a7f56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea8f562a1f24439c8732285d2fa1588e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:56.847342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-66c48ca3-c482-4071-a5ea-366e698a7f56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea8f562a1f24439c8732285d2fa1588e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:56.863308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-66c48ca3-c482-4071-a5ea-366e698a7f56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea8f562a1f24439c8732285d2fa1588e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:56.892256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1243/2490] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:30:56 2022] DELETE /identity/v3/users/ea8f562a1f24439c8732285d2fa1588e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:30:57.199522 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b39756-b3ce-418b-89e9-293310f4e95d admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:57.200608 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-72b39756-b3ce-418b-89e9-293310f4e95d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:57.219095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b39756-b3ce-418b-89e9-293310f4e95d admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:57.304266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4ea4f-4dbc-4629-b979-236bcfc47369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:57.304988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4ea4f-4dbc-4629-b979-236bcfc47369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:57.305792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4ea4f-4dbc-4629-b979-236bcfc47369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:57.306069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4ea4f-4dbc-4629-b979-236bcfc47369 admin admin] PATH_INFO: `/v3/projects/58b0fecf98bc490c9cb6937e9bb917bd` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:57.307314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4ea4f-4dbc-4629-b979-236bcfc47369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58b0fecf98bc490c9cb6937e9bb917bd)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:57.310048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4ea4f-4dbc-4629-b979-236bcfc47369 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:57.365854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-08b4ea4f-4dbc-4629-b979-236bcfc47369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58b0fecf98bc490c9cb6937e9bb917bd', 'request_id': 'req-08b4ea4f-4dbc-4629-b979-236bcfc47369'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:57.376626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-08b4ea4f-4dbc-4629-b979-236bcfc47369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58b0fecf98bc490c9cb6937e9bb917bd', 'request_id': 'req-08b4ea4f-4dbc-4629-b979-236bcfc47369'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:57.383629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-08b4ea4f-4dbc-4629-b979-236bcfc47369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58b0fecf98bc490c9cb6937e9bb917bd', 'request_id': 'req-08b4ea4f-4dbc-4629-b979-236bcfc47369'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:57.398535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-08b4ea4f-4dbc-4629-b979-236bcfc47369 admin admin] The token cache is being invalidate because project 58b0fecf98bc490c9cb6937e9bb917bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:30:57.399407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-08b4ea4f-4dbc-4629-b979-236bcfc47369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:57.400687 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-08b4ea4f-4dbc-4629-b979-236bcfc47369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:57.402515 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1248/2491] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:30:57 2022] DELETE /identity/v3/projects/58b0fecf98bc490c9cb6937e9bb917bd => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:30:57.407239 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb4a20-9672-43f3-9db2-ea6f981affbc None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:57.407558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb4a20-9672-43f3-9db2-ea6f981affbc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:57.407839 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb4a20-9672-43f3-9db2-ea6f981affbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:57.428296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ddcb4a20-9672-43f3-9db2-ea6f981affbc None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:57.429589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ddcb4a20-9672-43f3-9db2-ea6f981affbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:57.491617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1244/2492] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:30:57.500204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208128c9-ddf5-4126-b742-50d90c796864 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:57.500519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208128c9-ddf5-4126-b742-50d90c796864 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:57.500864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208128c9-ddf5-4126-b742-50d90c796864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:57.521499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-208128c9-ddf5-4126-b742-50d90c796864 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:57.522733 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-208128c9-ddf5-4126-b742-50d90c796864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:57.531027 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1249/2493] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:30:57.535331 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb4a20-9672-43f3-9db2-ea6f981affbc None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:57.546746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb4a20-9672-43f3-9db2-ea6f981affbc None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:57.551806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d8686-5aef-4768-bfbf-5f593932fe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:57.552485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d8686-5aef-4768-bfbf-5f593932fe37 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:57.552833 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d8686-5aef-4768-bfbf-5f593932fe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:57.553174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d8686-5aef-4768-bfbf-5f593932fe37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:57.553898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d8686-5aef-4768-bfbf-5f593932fe37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:30:57.554267 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d8686-5aef-4768-bfbf-5f593932fe37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:57.556517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d8686-5aef-4768-bfbf-5f593932fe37 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:57.568887 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1245/2494] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:30:57.573433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208128c9-ddf5-4126-b742-50d90c796864 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:57.582957 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208128c9-ddf5-4126-b742-50d90c796864 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:57.588271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31dac0-6389-486c-9186-0d242dcaab08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:57.588876 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31dac0-6389-486c-9186-0d242dcaab08 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:57.589156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31dac0-6389-486c-9186-0d242dcaab08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:57.589432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31dac0-6389-486c-9186-0d242dcaab08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:57.590071 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31dac0-6389-486c-9186-0d242dcaab08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:57.592037 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31dac0-6389-486c-9186-0d242dcaab08 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:57.604308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:57.622621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1250/2495] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:30:57 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:30:57.628539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d8686-5aef-4768-bfbf-5f593932fe37 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:57.639832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d8686-5aef-4768-bfbf-5f593932fe37 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:57.700296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12a1e4-363b-49cc-bb4a-84eda3e2e87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:57.700907 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12a1e4-363b-49cc-bb4a-84eda3e2e87e admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:57.701186 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12a1e4-363b-49cc-bb4a-84eda3e2e87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:57.701461 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12a1e4-363b-49cc-bb4a-84eda3e2e87e admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:57.702101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12a1e4-363b-49cc-bb4a-84eda3e2e87e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:57.704052 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12a1e4-363b-49cc-bb4a-84eda3e2e87e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:57.731132 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1246/2496] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:30:57 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:30:57.732168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31dac0-6389-486c-9186-0d242dcaab08 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:57.743623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31dac0-6389-486c-9186-0d242dcaab08 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:57.750016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40d6a1-3ff2-4841-a109-ff276ad1bddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:57.750632 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40d6a1-3ff2-4841-a109-ff276ad1bddf admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:57.750923 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40d6a1-3ff2-4841-a109-ff276ad1bddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:57.751202 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40d6a1-3ff2-4841-a109-ff276ad1bddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:57.752128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e40d6a1-3ff2-4841-a109-ff276ad1bddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:57.753772 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e40d6a1-3ff2-4841-a109-ff276ad1bddf admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:57.767906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1251/2497] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:30:57.769576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12a1e4-363b-49cc-bb4a-84eda3e2e87e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:57.780237 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12a1e4-363b-49cc-bb4a-84eda3e2e87e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:57.785016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851075fc-8121-4825-b230-af7cbb4c5298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:57.785649 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851075fc-8121-4825-b230-af7cbb4c5298 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:57.785926 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851075fc-8121-4825-b230-af7cbb4c5298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:57.786205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851075fc-8121-4825-b230-af7cbb4c5298 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:57.786844 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851075fc-8121-4825-b230-af7cbb4c5298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:57.788571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851075fc-8121-4825-b230-af7cbb4c5298 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:57.790894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:57.803739 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1247/2498] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:30:57.806335 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40d6a1-3ff2-4841-a109-ff276ad1bddf admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:57.818578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40d6a1-3ff2-4841-a109-ff276ad1bddf admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:57.824034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a59c9-2460-4c9d-a603-f77477bb0e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:57.824693 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a59c9-2460-4c9d-a603-f77477bb0e7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:57.825088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a59c9-2460-4c9d-a603-f77477bb0e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:57.825370 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a59c9-2460-4c9d-a603-f77477bb0e7a admin admin] PATH_INFO: `/v3/projects/a43894942aa24ae3995f9c8521725911/users/37c70e43dfd449498d71a68109898406/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:57.841231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a59c9-2460-4c9d-a603-f77477bb0e7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a43894942aa24ae3995f9c8521725911, user_id=37c70e43dfd449498d71a68109898406, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:57.843936 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a59c9-2460-4c9d-a603-f77477bb0e7a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:57.857507 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1252/2499] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:30:57 2022] PUT /identity/v3/projects/a43894942aa24ae3995f9c8521725911/users/37c70e43dfd449498d71a68109898406/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:30:57.868656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d45639-e4d1-486e-80f7-41aec9398c5e None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:57.868945 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d45639-e4d1-486e-80f7-41aec9398c5e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:57.869218 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d45639-e4d1-486e-80f7-41aec9398c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:57.898517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f6d45639-e4d1-486e-80f7-41aec9398c5e None None] MFA Rules not processed for user `37c70e43dfd449498d71a68109898406`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:57.899912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f6d45639-e4d1-486e-80f7-41aec9398c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:57.966383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1248/2500] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:30:57.971052 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a59c9-2460-4c9d-a603-f77477bb0e7a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:58.033875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a59c9-2460-4c9d-a603-f77477bb0e7a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:58.038537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d51e9a-8a4e-4d03-b9ad-425155fbdd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:58.039138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d51e9a-8a4e-4d03-b9ad-425155fbdd48 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:58.039445 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d51e9a-8a4e-4d03-b9ad-425155fbdd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:58.039746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d51e9a-8a4e-4d03-b9ad-425155fbdd48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:58.040371 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d51e9a-8a4e-4d03-b9ad-425155fbdd48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:58.042335 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d51e9a-8a4e-4d03-b9ad-425155fbdd48 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:58.069553 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1253/2501] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:30:57 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:30:58.074199 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d45639-e4d1-486e-80f7-41aec9398c5e None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:58.085321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d45639-e4d1-486e-80f7-41aec9398c5e None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:58.144750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0247cdf-e4b5-4d86-8152-c2e2e5694624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:58.145737 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0247cdf-e4b5-4d86-8152-c2e2e5694624 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:58.146014 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0247cdf-e4b5-4d86-8152-c2e2e5694624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:58.146355 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0247cdf-e4b5-4d86-8152-c2e2e5694624 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:58.146994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0247cdf-e4b5-4d86-8152-c2e2e5694624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:58.149582 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0247cdf-e4b5-4d86-8152-c2e2e5694624 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:58.171443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1249/2502] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:30:58 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:30:58.175470 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d51e9a-8a4e-4d03-b9ad-425155fbdd48 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:58.186506 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d51e9a-8a4e-4d03-b9ad-425155fbdd48 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:58.191385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bc889-b2e0-4f0e-8714-213d632ffc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:58.192039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bc889-b2e0-4f0e-8714-213d632ffc54 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:58.192317 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bc889-b2e0-4f0e-8714-213d632ffc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:58.192588 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bc889-b2e0-4f0e-8714-213d632ffc54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:58.193226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2bc889-b2e0-4f0e-8714-213d632ffc54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:58.194801 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2bc889-b2e0-4f0e-8714-213d632ffc54 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:58.206207 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1254/2503] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:30:58.210532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0247cdf-e4b5-4d86-8152-c2e2e5694624 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:58.221359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0247cdf-e4b5-4d86-8152-c2e2e5694624 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:58.226087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20385bf-e99d-4f10-af34-f0946e025361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:58.226760 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20385bf-e99d-4f10-af34-f0946e025361 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:58.227023 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20385bf-e99d-4f10-af34-f0946e025361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:58.227300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20385bf-e99d-4f10-af34-f0946e025361 admin admin] PATH_INFO: `/v3/projects/4688e0c380fe430bb52c5f2c372f5d5d/users/6e8fb16ca6624d9d9da3457c6404f7dc/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:58.253984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20385bf-e99d-4f10-af34-f0946e025361 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4688e0c380fe430bb52c5f2c372f5d5d, user_id=6e8fb16ca6624d9d9da3457c6404f7dc, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:58.257257 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20385bf-e99d-4f10-af34-f0946e025361 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:58.275717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1250/2504] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:30:58 2022] PUT /identity/v3/projects/4688e0c380fe430bb52c5f2c372f5d5d/users/6e8fb16ca6624d9d9da3457c6404f7dc/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:30:58.282185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bc889-b2e0-4f0e-8714-213d632ffc54 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:58.299492 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bc889-b2e0-4f0e-8714-213d632ffc54 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:58.350070 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b99f53-e5a7-4fb7-b73f-b19ef13fb2e1 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:58.350630 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b99f53-e5a7-4fb7-b73f-b19ef13fb2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:58.350985 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b99f53-e5a7-4fb7-b73f-b19ef13fb2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:58.366443 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8772e-857d-4c8f-89ed-d87dcf119121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:58.367060 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8772e-857d-4c8f-89ed-d87dcf119121 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:58.367339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8772e-857d-4c8f-89ed-d87dcf119121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:58.367834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8772e-857d-4c8f-89ed-d87dcf119121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:58.368482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8772e-857d-4c8f-89ed-d87dcf119121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:58.370251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8772e-857d-4c8f-89ed-d87dcf119121 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:58.381061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1255/2505] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:30:58.383102 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-26b99f53-e5a7-4fb7-b73f-b19ef13fb2e1 None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:58.384461 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-26b99f53-e5a7-4fb7-b73f-b19ef13fb2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:58.384958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8772e-857d-4c8f-89ed-d87dcf119121 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:58.396034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8772e-857d-4c8f-89ed-d87dcf119121 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:58.401841 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ee140-4714-4904-a689-930e377a57f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:58.402570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ee140-4714-4904-a689-930e377a57f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:58.403033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ee140-4714-4904-a689-930e377a57f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:58.403311 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ee140-4714-4904-a689-930e377a57f6 admin admin] PATH_INFO: `/v3/projects/4688e0c380fe430bb52c5f2c372f5d5d/users/6e8fb16ca6624d9d9da3457c6404f7dc/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:58.405290 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ee140-4714-4904-a689-930e377a57f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4688e0c380fe430bb52c5f2c372f5d5d, user_id=6e8fb16ca6624d9d9da3457c6404f7dc, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:58.407968 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ee140-4714-4904-a689-930e377a57f6 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:58.419886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1256/2506] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:30:58 2022] PUT /identity/v3/projects/4688e0c380fe430bb52c5f2c372f5d5d/users/6e8fb16ca6624d9d9da3457c6404f7dc/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:30:58.431406 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc6ebe-e693-46f2-aa40-20b495adc209 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:58.431745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc6ebe-e693-46f2-aa40-20b495adc209 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:58.432024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc6ebe-e693-46f2-aa40-20b495adc209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:58.432191 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:58.450013 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1251/2507] 10.222.0.33 () {58 vars in 967 bytes} [Thu Oct 6 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:30:58.461963 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:58.465494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-15dc6ebe-e693-46f2-aa40-20b495adc209 None None] MFA Rules not processed for user `6e8fb16ca6624d9d9da3457c6404f7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:58.468816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-15dc6ebe-e693-46f2-aa40-20b495adc209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:58.510659 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b99f53-e5a7-4fb7-b73f-b19ef13fb2e1 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:58.512237 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-26b99f53-e5a7-4fb7-b73f-b19ef13fb2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:58.546969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1257/2508] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:30:58.561071 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc6ebe-e693-46f2-aa40-20b495adc209 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:58.563249 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-15dc6ebe-e693-46f2-aa40-20b495adc209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:58.593058 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b99f53-e5a7-4fb7-b73f-b19ef13fb2e1 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:58.625727 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc6ebe-e693-46f2-aa40-20b495adc209 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:58.630610 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6804a77-b9d7-4cc9-827d-ae13d5e1f1e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:58.631226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6804a77-b9d7-4cc9-827d-ae13d5e1f1e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:58.631558 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6804a77-b9d7-4cc9-827d-ae13d5e1f1e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:58.631817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6804a77-b9d7-4cc9-827d-ae13d5e1f1e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:58.642360 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6804a77-b9d7-4cc9-827d-ae13d5e1f1e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:58.643828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6804a77-b9d7-4cc9-827d-ae13d5e1f1e0 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:58.654537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1258/2509] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:30:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:30:58.660717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1ff78-961e-414e-badd-83962c62f18b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:58.661412 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1ff78-961e-414e-badd-83962c62f18b service swift] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:58.661836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1ff78-961e-414e-badd-83962c62f18b service swift] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:58.662271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1ff78-961e-414e-badd-83962c62f18b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:58.730200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1ff78-961e-414e-badd-83962c62f18b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:58.731997 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1ff78-961e-414e-badd-83962c62f18b service swift] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:58.740986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1252/2510] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:30:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 06 12:30:59.068704 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6804a77-b9d7-4cc9-827d-ae13d5e1f1e0 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:59.078434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6804a77-b9d7-4cc9-827d-ae13d5e1f1e0 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:59.083181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56572d0b-644f-4b19-a100-e5ccedbf8c18 tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e8fb16ca6624d9d9da3457c6404f7dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4688e0c380fe430bb52c5f2c372f5d5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:59.084189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56572d0b-644f-4b19-a100-e5ccedbf8c18 tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:59.084638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56572d0b-644f-4b19-a100-e5ccedbf8c18 tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:59.084985 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56572d0b-644f-4b19-a100-e5ccedbf8c18 tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:59.085992 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56572d0b-644f-4b19-a100-e5ccedbf8c18 tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:30:59.086982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56572d0b-644f-4b19-a100-e5ccedbf8c18 tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:59.088982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56572d0b-644f-4b19-a100-e5ccedbf8c18 tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:59.101475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1259/2511] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:30:59.106222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1ff78-961e-414e-badd-83962c62f18b service swift] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:59.119533 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1ff78-961e-414e-badd-83962c62f18b service swift] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:59.126661 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc54954-03e7-4ced-8517-5b649eb28f7c tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e8fb16ca6624d9d9da3457c6404f7dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4688e0c380fe430bb52c5f2c372f5d5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:59.127720 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc54954-03e7-4ced-8517-5b649eb28f7c tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:59.128383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc54954-03e7-4ced-8517-5b649eb28f7c tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:59.128866 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc54954-03e7-4ced-8517-5b649eb28f7c tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:59.129665 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc54954-03e7-4ced-8517-5b649eb28f7c tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:59.131733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc54954-03e7-4ced-8517-5b649eb28f7c tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:59.153201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56572d0b-644f-4b19-a100-e5ccedbf8c18 tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:59.155583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-56572d0b-644f-4b19-a100-e5ccedbf8c18 tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:59.166773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1253/2512] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:30:59 2022] POST /identity/v3/projects => generated 354 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:30:59.170593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc54954-03e7-4ced-8517-5b649eb28f7c tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:59.183943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:59.220066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56572d0b-644f-4b19-a100-e5ccedbf8c18 tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:59.236627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc54954-03e7-4ced-8517-5b649eb28f7c tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:59.241504 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da19bb-c0c9-4144-ba33-0a8a02da99ce tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e8fb16ca6624d9d9da3457c6404f7dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4688e0c380fe430bb52c5f2c372f5d5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:59.241886 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da19bb-c0c9-4144-ba33-0a8a02da99ce tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:59.242172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da19bb-c0c9-4144-ba33-0a8a02da99ce tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:59.242963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da19bb-c0c9-4144-ba33-0a8a02da99ce tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:59.243642 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da19bb-c0c9-4144-ba33-0a8a02da99ce tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:30:59.243947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da19bb-c0c9-4144-ba33-0a8a02da99ce tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:59.245542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da19bb-c0c9-4144-ba33-0a8a02da99ce tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:59.258975 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1254/2513] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:30:59.273389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:30:59.283884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29051029-e6f7-496e-9a66-8486c64f4e3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:59.284651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29051029-e6f7-496e-9a66-8486c64f4e3b service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:59.284992 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29051029-e6f7-496e-9a66-8486c64f4e3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:59.285417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29051029-e6f7-496e-9a66-8486c64f4e3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:59.288192 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-29051029-e6f7-496e-9a66-8486c64f4e3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:59.313292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da19bb-c0c9-4144-ba33-0a8a02da99ce tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:59.330477 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da19bb-c0c9-4144-ba33-0a8a02da99ce tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:59.335923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d53c9-43bc-4316-a121-5ca4a8c04b6f tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e8fb16ca6624d9d9da3457c6404f7dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4688e0c380fe430bb52c5f2c372f5d5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:59.336725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d53c9-43bc-4316-a121-5ca4a8c04b6f tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:59.337011 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d53c9-43bc-4316-a121-5ca4a8c04b6f tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:59.337289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d53c9-43bc-4316-a121-5ca4a8c04b6f tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] PATH_INFO: `/v3/projects/f3776f5c9f644bd698e414e2b4604919` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:59.338463 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683d53c9-43bc-4316-a121-5ca4a8c04b6f tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f3776f5c9f644bd698e414e2b4604919)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:59.340427 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683d53c9-43bc-4316-a121-5ca4a8c04b6f tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:59.368173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29051029-e6f7-496e-9a66-8486c64f4e3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:59.370043 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29051029-e6f7-496e-9a66-8486c64f4e3b service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:59.381633 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1260/2514] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:30:59.393156 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-683d53c9-43bc-4316-a121-5ca4a8c04b6f tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3776f5c9f644bd698e414e2b4604919', 'request_id': 'req-683d53c9-43bc-4316-a121-5ca4a8c04b6f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:59.402807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-683d53c9-43bc-4316-a121-5ca4a8c04b6f tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3776f5c9f644bd698e414e2b4604919', 'request_id': 'req-683d53c9-43bc-4316-a121-5ca4a8c04b6f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:59.409855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-683d53c9-43bc-4316-a121-5ca4a8c04b6f tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3776f5c9f644bd698e414e2b4604919', 'request_id': 'req-683d53c9-43bc-4316-a121-5ca4a8c04b6f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:59.420738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-683d53c9-43bc-4316-a121-5ca4a8c04b6f tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] The token cache is being invalidate because project f3776f5c9f644bd698e414e2b4604919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:30:59.421034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-683d53c9-43bc-4316-a121-5ca4a8c04b6f tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:59.422144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-683d53c9-43bc-4316-a121-5ca4a8c04b6f tempest-VolumeQuotaClassesTest-1656232006 tempest-VolumeQuotaClassesTest-1656232006-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:59.423707 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1255/2515] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:30:59 2022] DELETE /identity/v3/projects/f3776f5c9f644bd698e414e2b4604919 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:30:59.657104 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29051029-e6f7-496e-9a66-8486c64f4e3b service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:59.667175 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29051029-e6f7-496e-9a66-8486c64f4e3b service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:30:59.728307 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5515f-beef-461a-a240-fd862a49b508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:30:59.729090 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5515f-beef-461a-a240-fd862a49b508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:59.729935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5515f-beef-461a-a240-fd862a49b508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:59.730290 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5515f-beef-461a-a240-fd862a49b508 admin admin] PATH_INFO: `/v3/users/37c70e43dfd449498d71a68109898406` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:59.731521 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e5515f-beef-461a-a240-fd862a49b508 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37c70e43dfd449498d71a68109898406)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:30:59.733560 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e5515f-beef-461a-a240-fd862a49b508 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:30:59.794883 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e1e5515f-beef-461a-a240-fd862a49b508 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37c70e43dfd449498d71a68109898406'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:59.805087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e1e5515f-beef-461a-a240-fd862a49b508 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37c70e43dfd449498d71a68109898406'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:59.805978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e1e5515f-beef-461a-a240-fd862a49b508 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37c70e43dfd449498d71a68109898406'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:59.806926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e1e5515f-beef-461a-a240-fd862a49b508 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37c70e43dfd449498d71a68109898406'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:59.818374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-e1e5515f-beef-461a-a240-fd862a49b508 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37c70e43dfd449498d71a68109898406'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:30:59.846089 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1261/2516] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:30:59 2022] DELETE /identity/v3/users/37c70e43dfd449498d71a68109898406 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:30:59.849309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac64c19-ec27-4822-a8d0-e9db1b31a907 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:30:59.849309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac64c19-ec27-4822-a8d0-e9db1b31a907 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:30:59.849309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac64c19-ec27-4822-a8d0-e9db1b31a907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:30:59.869841 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8ac64c19-ec27-4822-a8d0-e9db1b31a907 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:30:59.871483 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8ac64c19-ec27-4822-a8d0-e9db1b31a907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:59.942152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1256/2517] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:30:59.953985 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5515f-beef-461a-a240-fd862a49b508 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:30:59.956149 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e1e5515f-beef-461a-a240-fd862a49b508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:30:59.976018 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:00.025975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5515f-beef-461a-a240-fd862a49b508 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:00.029956 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9acbc5-401e-47d1-9405-ac69644ff729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:00.030639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9acbc5-401e-47d1-9405-ac69644ff729 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:00.031002 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9acbc5-401e-47d1-9405-ac69644ff729 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:00.031459 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9acbc5-401e-47d1-9405-ac69644ff729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:00.041469 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9acbc5-401e-47d1-9405-ac69644ff729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:00.042949 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9acbc5-401e-47d1-9405-ac69644ff729 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:00.053808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1262/2518] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:31:00.341368 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac64c19-ec27-4822-a8d0-e9db1b31a907 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:00.346108 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8ac64c19-ec27-4822-a8d0-e9db1b31a907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:00.365872 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac64c19-ec27-4822-a8d0-e9db1b31a907 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:00.378530 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facedee0-2285-4d84-8210-ef191e50e838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:00.379203 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facedee0-2285-4d84-8210-ef191e50e838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:00.379943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facedee0-2285-4d84-8210-ef191e50e838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:00.380236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facedee0-2285-4d84-8210-ef191e50e838 admin admin] PATH_INFO: `/v3/projects/a43894942aa24ae3995f9c8521725911` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:00.381484 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facedee0-2285-4d84-8210-ef191e50e838 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a43894942aa24ae3995f9c8521725911)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:00.383446 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facedee0-2285-4d84-8210-ef191e50e838 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:00.411738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:00.442034 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-facedee0-2285-4d84-8210-ef191e50e838 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a43894942aa24ae3995f9c8521725911', 'request_id': 'req-facedee0-2285-4d84-8210-ef191e50e838'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:00.450182 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-facedee0-2285-4d84-8210-ef191e50e838 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a43894942aa24ae3995f9c8521725911', 'request_id': 'req-facedee0-2285-4d84-8210-ef191e50e838'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:00.457217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-facedee0-2285-4d84-8210-ef191e50e838 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a43894942aa24ae3995f9c8521725911', 'request_id': 'req-facedee0-2285-4d84-8210-ef191e50e838'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:00.469210 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-facedee0-2285-4d84-8210-ef191e50e838 admin admin] The token cache is being invalidate because project a43894942aa24ae3995f9c8521725911 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:31:00.469502 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-facedee0-2285-4d84-8210-ef191e50e838 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:00.470167 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-facedee0-2285-4d84-8210-ef191e50e838 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:00.471817 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1257/2519] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:31:00 2022] DELETE /identity/v3/projects/a43894942aa24ae3995f9c8521725911 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:31:00.476102 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9acbc5-401e-47d1-9405-ac69644ff729 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:00.487263 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9acbc5-401e-47d1-9405-ac69644ff729 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:00.559029 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6dd21-8fc7-4f24-9a21-562ca93e8558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:00.559739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6dd21-8fc7-4f24-9a21-562ca93e8558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:00.559985 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6dd21-8fc7-4f24-9a21-562ca93e8558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:00.560278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6dd21-8fc7-4f24-9a21-562ca93e8558 admin admin] PATH_INFO: `/v3/users/6e8fb16ca6624d9d9da3457c6404f7dc` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:00.561416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6dd21-8fc7-4f24-9a21-562ca93e8558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8fb16ca6624d9d9da3457c6404f7dc)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:00.563322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6dd21-8fc7-4f24-9a21-562ca93e8558 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:00.650478 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-4fc6dd21-8fc7-4f24-9a21-562ca93e8558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8fb16ca6624d9d9da3457c6404f7dc'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:00.670429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-4fc6dd21-8fc7-4f24-9a21-562ca93e8558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8fb16ca6624d9d9da3457c6404f7dc'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:00.672267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-4fc6dd21-8fc7-4f24-9a21-562ca93e8558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8fb16ca6624d9d9da3457c6404f7dc'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:00.674671 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-4fc6dd21-8fc7-4f24-9a21-562ca93e8558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8fb16ca6624d9d9da3457c6404f7dc'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:00.704144 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-4fc6dd21-8fc7-4f24-9a21-562ca93e8558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8fb16ca6624d9d9da3457c6404f7dc'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:00.753180 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1263/2520] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:31:00 2022] DELETE /identity/v3/users/6e8fb16ca6624d9d9da3457c6404f7dc => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:01.032482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facedee0-2285-4d84-8210-ef191e50e838 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:01.034895 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-facedee0-2285-4d84-8210-ef191e50e838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:01.060687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facedee0-2285-4d84-8210-ef191e50e838 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:01.275344 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c0003-0b9a-4774-8006-08c0d0ce7c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:01.276030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c0003-0b9a-4774-8006-08c0d0ce7c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:01.276315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c0003-0b9a-4774-8006-08c0d0ce7c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:01.276590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c0003-0b9a-4774-8006-08c0d0ce7c2d admin admin] PATH_INFO: `/v3/projects/4688e0c380fe430bb52c5f2c372f5d5d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:01.277871 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c0003-0b9a-4774-8006-08c0d0ce7c2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4688e0c380fe430bb52c5f2c372f5d5d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:01.279939 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c0003-0b9a-4774-8006-08c0d0ce7c2d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:01.336556 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5a4c0003-0b9a-4774-8006-08c0d0ce7c2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4688e0c380fe430bb52c5f2c372f5d5d', 'request_id': 'req-5a4c0003-0b9a-4774-8006-08c0d0ce7c2d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:01.346191 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5a4c0003-0b9a-4774-8006-08c0d0ce7c2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4688e0c380fe430bb52c5f2c372f5d5d', 'request_id': 'req-5a4c0003-0b9a-4774-8006-08c0d0ce7c2d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:01.358380 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5a4c0003-0b9a-4774-8006-08c0d0ce7c2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4688e0c380fe430bb52c5f2c372f5d5d', 'request_id': 'req-5a4c0003-0b9a-4774-8006-08c0d0ce7c2d'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:01.370546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5a4c0003-0b9a-4774-8006-08c0d0ce7c2d admin admin] The token cache is being invalidate because project 4688e0c380fe430bb52c5f2c372f5d5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:31:01.370834 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5a4c0003-0b9a-4774-8006-08c0d0ce7c2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:01.371550 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-5a4c0003-0b9a-4774-8006-08c0d0ce7c2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:01.373260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1258/2521] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:31:01 2022] DELETE /identity/v3/projects/4688e0c380fe430bb52c5f2c372f5d5d => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:01.378050 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d76ba-2bfc-4931-84a2-bd4b43cd73c5 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:01.378490 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d76ba-2bfc-4931-84a2-bd4b43cd73c5 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:01.378770 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d76ba-2bfc-4931-84a2-bd4b43cd73c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:01.406983 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-fe6d76ba-2bfc-4931-84a2-bd4b43cd73c5 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:01.408325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fe6d76ba-2bfc-4931-84a2-bd4b43cd73c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:01.440284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:01.478998 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1264/2522] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:01.487313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00304a-a355-404b-b269-081e489994ac None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:01.487640 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00304a-a355-404b-b269-081e489994ac None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:01.487923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00304a-a355-404b-b269-081e489994ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:01.508354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-5c00304a-a355-404b-b269-081e489994ac None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:01.509567 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5c00304a-a355-404b-b269-081e489994ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:01.518847 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1259/2523] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:01.523189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d76ba-2bfc-4931-84a2-bd4b43cd73c5 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:01.533451 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d76ba-2bfc-4931-84a2-bd4b43cd73c5 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:01.538155 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3cbbe-f6e8-454d-a9bf-b9d611193904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:01.538764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3cbbe-f6e8-454d-a9bf-b9d611193904 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:01.539093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3cbbe-f6e8-454d-a9bf-b9d611193904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:01.539896 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3cbbe-f6e8-454d-a9bf-b9d611193904 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:01.540541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3cbbe-f6e8-454d-a9bf-b9d611193904 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:31:01.540843 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3cbbe-f6e8-454d-a9bf-b9d611193904 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:01.542440 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3cbbe-f6e8-454d-a9bf-b9d611193904 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:01.555426 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1265/2524] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:31:01.560142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00304a-a355-404b-b269-081e489994ac None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:01.572251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00304a-a355-404b-b269-081e489994ac None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:01.578451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47524a07-c781-4eb1-b322-b993fb38ed5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:01.579104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47524a07-c781-4eb1-b322-b993fb38ed5b admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:01.579380 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47524a07-c781-4eb1-b322-b993fb38ed5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:01.580615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47524a07-c781-4eb1-b322-b993fb38ed5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:01.581278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47524a07-c781-4eb1-b322-b993fb38ed5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:01.583182 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47524a07-c781-4eb1-b322-b993fb38ed5b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:01.613522 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1260/2525] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:31:01 2022] POST /identity/v3/projects => generated 401 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:31:01.617893 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3cbbe-f6e8-454d-a9bf-b9d611193904 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:01.628947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3cbbe-f6e8-454d-a9bf-b9d611193904 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:01.704503 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e099e1-fbd7-4d05-a398-8ca7a820dcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:01.705200 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e099e1-fbd7-4d05-a398-8ca7a820dcf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:01.705554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e099e1-fbd7-4d05-a398-8ca7a820dcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:01.705899 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e099e1-fbd7-4d05-a398-8ca7a820dcf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:01.706606 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e099e1-fbd7-4d05-a398-8ca7a820dcf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:01.708698 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e099e1-fbd7-4d05-a398-8ca7a820dcf8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:01.732857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1266/2526] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:31:01 2022] POST /identity/v3/users => generated 301 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:31:01.736815 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47524a07-c781-4eb1-b322-b993fb38ed5b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:01.746958 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47524a07-c781-4eb1-b322-b993fb38ed5b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:01.751684 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82cfc2-ed17-43d4-bbd9-8c8bccc671e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:01.752287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82cfc2-ed17-43d4-bbd9-8c8bccc671e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:01.752566 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82cfc2-ed17-43d4-bbd9-8c8bccc671e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:01.752839 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82cfc2-ed17-43d4-bbd9-8c8bccc671e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:01.753480 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82cfc2-ed17-43d4-bbd9-8c8bccc671e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:01.755060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82cfc2-ed17-43d4-bbd9-8c8bccc671e2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:01.766037 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1261/2527] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:31:01.770126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e099e1-fbd7-4d05-a398-8ca7a820dcf8 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:01.781531 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e099e1-fbd7-4d05-a398-8ca7a820dcf8 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:01.786346 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6eb119-865a-4bb1-9868-18626bbdda21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:01.787020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6eb119-865a-4bb1-9868-18626bbdda21 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:01.787374 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6eb119-865a-4bb1-9868-18626bbdda21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:01.787745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6eb119-865a-4bb1-9868-18626bbdda21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:01.788461 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6eb119-865a-4bb1-9868-18626bbdda21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:01.790155 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6eb119-865a-4bb1-9868-18626bbdda21 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:01.803148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1267/2528] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:31:01.806916 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82cfc2-ed17-43d4-bbd9-8c8bccc671e2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:01.812399 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:01.821079 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82cfc2-ed17-43d4-bbd9-8c8bccc671e2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:01.827955 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fcc78-c6dc-4a21-b643-051d77855e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:01.828699 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fcc78-c6dc-4a21-b643-051d77855e0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:01.829048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fcc78-c6dc-4a21-b643-051d77855e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:01.829418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fcc78-c6dc-4a21-b643-051d77855e0d admin admin] PATH_INFO: `/v3/projects/b9554a6be96c423bb5dd5125c2920a61/users/149a82ddc13c4e73a58d93d9f238ace9/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:01.845590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fcc78-c6dc-4a21-b643-051d77855e0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9554a6be96c423bb5dd5125c2920a61, user_id=149a82ddc13c4e73a58d93d9f238ace9, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:01.848254 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fcc78-c6dc-4a21-b643-051d77855e0d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:01.863541 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1262/2529] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:31:01 2022] PUT /identity/v3/projects/b9554a6be96c423bb5dd5125c2920a61/users/149a82ddc13c4e73a58d93d9f238ace9/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:01.874554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c379a5-b7ca-4dbd-a122-5aac6e3fb857 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:01.874554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c379a5-b7ca-4dbd-a122-5aac6e3fb857 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:01.874554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c379a5-b7ca-4dbd-a122-5aac6e3fb857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:01.906705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d4c379a5-b7ca-4dbd-a122-5aac6e3fb857 None None] MFA Rules not processed for user `149a82ddc13c4e73a58d93d9f238ace9`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:01.908031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d4c379a5-b7ca-4dbd-a122-5aac6e3fb857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:01.986857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1268/2530] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:01.991424 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fcc78-c6dc-4a21-b643-051d77855e0d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:02.084606 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fcc78-c6dc-4a21-b643-051d77855e0d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:02.094681 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba694c4-64c9-4fbf-ad9b-f06b88aad98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:02.096190 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba694c4-64c9-4fbf-ad9b-f06b88aad98e admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:02.096542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba694c4-64c9-4fbf-ad9b-f06b88aad98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:02.096911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba694c4-64c9-4fbf-ad9b-f06b88aad98e admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:02.097653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba694c4-64c9-4fbf-ad9b-f06b88aad98e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:02.099711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba694c4-64c9-4fbf-ad9b-f06b88aad98e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:02.135734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1263/2531] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:31:01 2022] POST /identity/v3/projects => generated 397 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:31:02.136928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c379a5-b7ca-4dbd-a122-5aac6e3fb857 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:02.158202 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c379a5-b7ca-4dbd-a122-5aac6e3fb857 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:02.194306 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:02.227717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd18ec8-513e-4749-89c4-5df074c9cece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:02.228370 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd18ec8-513e-4749-89c4-5df074c9cece admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:02.228930 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd18ec8-513e-4749-89c4-5df074c9cece admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:02.229042 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd18ec8-513e-4749-89c4-5df074c9cece admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:02.229800 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd18ec8-513e-4749-89c4-5df074c9cece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:02.232460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd18ec8-513e-4749-89c4-5df074c9cece admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:02.264114 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1269/2532] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:31:02 2022] POST /identity/v3/users => generated 305 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:31:02.268769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba694c4-64c9-4fbf-ad9b-f06b88aad98e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:02.283380 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba694c4-64c9-4fbf-ad9b-f06b88aad98e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:02.289051 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfa73e-bd55-4541-b8ba-b1c9cf6288fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:02.290330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfa73e-bd55-4541-b8ba-b1c9cf6288fb admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:02.290678 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfa73e-bd55-4541-b8ba-b1c9cf6288fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:02.291037 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfa73e-bd55-4541-b8ba-b1c9cf6288fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:02.291851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfa73e-bd55-4541-b8ba-b1c9cf6288fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:02.293545 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfa73e-bd55-4541-b8ba-b1c9cf6288fb admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:02.308057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1264/2533] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:31:02.309586 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd18ec8-513e-4749-89c4-5df074c9cece admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:02.323647 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd18ec8-513e-4749-89c4-5df074c9cece admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:02.334049 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14262490-8ca1-4253-a675-ba0139e3be1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:02.334877 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14262490-8ca1-4253-a675-ba0139e3be1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:02.335264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14262490-8ca1-4253-a675-ba0139e3be1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:02.336613 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14262490-8ca1-4253-a675-ba0139e3be1a admin admin] PATH_INFO: `/v3/projects/4696979293fc426bb08adbb6162cdfa8/users/addd018c429643b3aaed4bb1aa4de9fa/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:02.365016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14262490-8ca1-4253-a675-ba0139e3be1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4696979293fc426bb08adbb6162cdfa8, user_id=addd018c429643b3aaed4bb1aa4de9fa, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:02.370028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14262490-8ca1-4253-a675-ba0139e3be1a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:02.385492 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1270/2534] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:31:02 2022] PUT /identity/v3/projects/4696979293fc426bb08adbb6162cdfa8/users/addd018c429643b3aaed4bb1aa4de9fa/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:02.390105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfa73e-bd55-4541-b8ba-b1c9cf6288fb admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:02.404446 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfa73e-bd55-4541-b8ba-b1c9cf6288fb admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:02.480703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cfe4d-1c58-4124-976b-f8a14306e3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:02.481314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cfe4d-1c58-4124-976b-f8a14306e3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:02.481593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cfe4d-1c58-4124-976b-f8a14306e3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:02.481856 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cfe4d-1c58-4124-976b-f8a14306e3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:02.482488 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cfe4d-1c58-4124-976b-f8a14306e3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:02.484741 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cfe4d-1c58-4124-976b-f8a14306e3b8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:02.503829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1265/2535] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:31:02.507893 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14262490-8ca1-4253-a675-ba0139e3be1a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:02.519850 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14262490-8ca1-4253-a675-ba0139e3be1a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:02.526535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df8ea4-586c-42a8-8686-4ae4b38724d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:02.527297 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df8ea4-586c-42a8-8686-4ae4b38724d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:02.527740 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df8ea4-586c-42a8-8686-4ae4b38724d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:02.528084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df8ea4-586c-42a8-8686-4ae4b38724d4 admin admin] PATH_INFO: `/v3/projects/4696979293fc426bb08adbb6162cdfa8/users/addd018c429643b3aaed4bb1aa4de9fa/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:02.530096 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df8ea4-586c-42a8-8686-4ae4b38724d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4696979293fc426bb08adbb6162cdfa8, user_id=addd018c429643b3aaed4bb1aa4de9fa, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:02.532847 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df8ea4-586c-42a8-8686-4ae4b38724d4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:02.546879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1271/2536] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:31:02 2022] PUT /identity/v3/projects/4696979293fc426bb08adbb6162cdfa8/users/addd018c429643b3aaed4bb1aa4de9fa/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:02.557819 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf59ba-1d88-4780-a248-e3d44c143b6b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:02.558191 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf59ba-1d88-4780-a248-e3d44c143b6b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:02.558534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf59ba-1d88-4780-a248-e3d44c143b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:02.602729 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-8baf59ba-1d88-4780-a248-e3d44c143b6b None None] MFA Rules not processed for user `addd018c429643b3aaed4bb1aa4de9fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:02.604332 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8baf59ba-1d88-4780-a248-e3d44c143b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:02.657640 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:02.694160 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1266/2537] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:02.715884 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df8ea4-586c-42a8-8686-4ae4b38724d4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:02.718827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-22df8ea4-586c-42a8-8686-4ae4b38724d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:02.795985 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf59ba-1d88-4780-a248-e3d44c143b6b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:02.797465 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-8baf59ba-1d88-4780-a248-e3d44c143b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:02.800096 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df8ea4-586c-42a8-8686-4ae4b38724d4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:02.803784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da50ef-bbd1-4e20-8ab0-a912532f67bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:02.804572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da50ef-bbd1-4e20-8ab0-a912532f67bb service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:02.805390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da50ef-bbd1-4e20-8ab0-a912532f67bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:02.805664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da50ef-bbd1-4e20-8ab0-a912532f67bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:02.819295 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da50ef-bbd1-4e20-8ab0-a912532f67bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:02.820816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da50ef-bbd1-4e20-8ab0-a912532f67bb service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:02.838309 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1272/2538] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:31:02.884927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf59ba-1d88-4780-a248-e3d44c143b6b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:02.893153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf7121-2246-48a4-94fd-a9d2c6490f6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:02.894005 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf7121-2246-48a4-94fd-a9d2c6490f6f service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:02.894424 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf7121-2246-48a4-94fd-a9d2c6490f6f service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:02.894805 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf7121-2246-48a4-94fd-a9d2c6490f6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:02.897614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3adf7121-2246-48a4-94fd-a9d2c6490f6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:02.911163 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf7121-2246-48a4-94fd-a9d2c6490f6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:02.912924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf7121-2246-48a4-94fd-a9d2c6490f6f service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:02.924164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da50ef-bbd1-4e20-8ab0-a912532f67bb service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:02.928240 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1267/2539] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:31:02.935734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da50ef-bbd1-4e20-8ab0-a912532f67bb service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:02.939627 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c1294-149b-45fb-9562-d54336805633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:02.940278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c1294-149b-45fb-9562-d54336805633 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:02.940567 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c1294-149b-45fb-9562-d54336805633 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:02.940823 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c1294-149b-45fb-9562-d54336805633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:03.008712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c1294-149b-45fb-9562-d54336805633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:03.010402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c1294-149b-45fb-9562-d54336805633 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:03.019634 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1273/2540] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:31:06.092553 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf7121-2246-48a4-94fd-a9d2c6490f6f service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:06.094029 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3adf7121-2246-48a4-94fd-a9d2c6490f6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:06.157735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf7121-2246-48a4-94fd-a9d2c6490f6f service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:06.162094 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1d60b-ef2b-437b-a0d5-09abcf969f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:06.162712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1d60b-ef2b-437b-a0d5-09abcf969f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:06.162987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1d60b-ef2b-437b-a0d5-09abcf969f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:06.163257 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1d60b-ef2b-437b-a0d5-09abcf969f9c admin admin] PATH_INFO: `/v3/users/196f71447aaf45fd812efb9eeba2c0cb` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:06.164434 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1d60b-ef2b-437b-a0d5-09abcf969f9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=196f71447aaf45fd812efb9eeba2c0cb)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:06.166334 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1d60b-ef2b-437b-a0d5-09abcf969f9c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:06.211768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3cc1d60b-ef2b-437b-a0d5-09abcf969f9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '196f71447aaf45fd812efb9eeba2c0cb'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:06.218621 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3cc1d60b-ef2b-437b-a0d5-09abcf969f9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '196f71447aaf45fd812efb9eeba2c0cb'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:06.219289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3cc1d60b-ef2b-437b-a0d5-09abcf969f9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '196f71447aaf45fd812efb9eeba2c0cb'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:06.219951 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3cc1d60b-ef2b-437b-a0d5-09abcf969f9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '196f71447aaf45fd812efb9eeba2c0cb'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:06.227965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3cc1d60b-ef2b-437b-a0d5-09abcf969f9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '196f71447aaf45fd812efb9eeba2c0cb'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:06.229801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:06.249785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1268/2541] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:31:06 2022] DELETE /identity/v3/users/196f71447aaf45fd812efb9eeba2c0cb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:06.642663 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c1294-149b-45fb-9562-d54336805633 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:06.658828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c1294-149b-45fb-9562-d54336805633 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:06.686006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:06.722564 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898eb468-915c-49cc-bb0c-f4e99172687b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:06.723183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898eb468-915c-49cc-bb0c-f4e99172687b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:06.723462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898eb468-915c-49cc-bb0c-f4e99172687b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:06.723831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898eb468-915c-49cc-bb0c-f4e99172687b admin admin] PATH_INFO: `/v3/projects/1c822fb7252b4804aa9a0cb228b6397c` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:06.724935 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898eb468-915c-49cc-bb0c-f4e99172687b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c822fb7252b4804aa9a0cb228b6397c)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:06.727425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898eb468-915c-49cc-bb0c-f4e99172687b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:06.773873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-898eb468-915c-49cc-bb0c-f4e99172687b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c822fb7252b4804aa9a0cb228b6397c', 'request_id': 'req-898eb468-915c-49cc-bb0c-f4e99172687b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:06.780638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-898eb468-915c-49cc-bb0c-f4e99172687b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c822fb7252b4804aa9a0cb228b6397c', 'request_id': 'req-898eb468-915c-49cc-bb0c-f4e99172687b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:06.786996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-898eb468-915c-49cc-bb0c-f4e99172687b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c822fb7252b4804aa9a0cb228b6397c', 'request_id': 'req-898eb468-915c-49cc-bb0c-f4e99172687b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:06.797715 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-898eb468-915c-49cc-bb0c-f4e99172687b admin admin] The token cache is being invalidate because project 1c822fb7252b4804aa9a0cb228b6397c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:31:06.797998 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-898eb468-915c-49cc-bb0c-f4e99172687b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:06.798622 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-898eb468-915c-49cc-bb0c-f4e99172687b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:06.800101 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1274/2542] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:31:06 2022] DELETE /identity/v3/projects/1c822fb7252b4804aa9a0cb228b6397c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:06.804081 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1d60b-ef2b-437b-a0d5-09abcf969f9c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:06.805551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3cc1d60b-ef2b-437b-a0d5-09abcf969f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:06.818411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1d60b-ef2b-437b-a0d5-09abcf969f9c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:06.875018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf20c95-5e62-44b6-94ac-44d3a76ef691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:06.875692 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf20c95-5e62-44b6-94ac-44d3a76ef691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:06.875970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf20c95-5e62-44b6-94ac-44d3a76ef691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:06.876240 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf20c95-5e62-44b6-94ac-44d3a76ef691 admin admin] PATH_INFO: `/v3/users/fccbeb90078246019c9e032e500b9df5` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:06.877401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf20c95-5e62-44b6-94ac-44d3a76ef691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fccbeb90078246019c9e032e500b9df5)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:06.879302 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf20c95-5e62-44b6-94ac-44d3a76ef691 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:06.924182 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-edf20c95-5e62-44b6-94ac-44d3a76ef691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fccbeb90078246019c9e032e500b9df5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:06.931028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-edf20c95-5e62-44b6-94ac-44d3a76ef691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fccbeb90078246019c9e032e500b9df5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:06.931711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-edf20c95-5e62-44b6-94ac-44d3a76ef691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fccbeb90078246019c9e032e500b9df5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:06.932321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-edf20c95-5e62-44b6-94ac-44d3a76ef691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fccbeb90078246019c9e032e500b9df5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:06.939469 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-edf20c95-5e62-44b6-94ac-44d3a76ef691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fccbeb90078246019c9e032e500b9df5'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:06.956245 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1269/2543] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:31:06 2022] DELETE /identity/v3/users/fccbeb90078246019c9e032e500b9df5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:07.138291 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898eb468-915c-49cc-bb0c-f4e99172687b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:07.139782 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-898eb468-915c-49cc-bb0c-f4e99172687b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:07.160369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898eb468-915c-49cc-bb0c-f4e99172687b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:07.232836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d5cef-fe09-4d15-b2b6-308dd2bd8f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:07.233477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d5cef-fe09-4d15-b2b6-308dd2bd8f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:07.233752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d5cef-fe09-4d15-b2b6-308dd2bd8f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:07.234022 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d5cef-fe09-4d15-b2b6-308dd2bd8f20 admin admin] PATH_INFO: `/v3/projects/b82af73e810a4053a0a125077f1b22b5` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:07.235149 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d5cef-fe09-4d15-b2b6-308dd2bd8f20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b82af73e810a4053a0a125077f1b22b5)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:07.237239 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d5cef-fe09-4d15-b2b6-308dd2bd8f20 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:07.278735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf20c95-5e62-44b6-94ac-44d3a76ef691 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:07.280832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-edf20c95-5e62-44b6-94ac-44d3a76ef691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:07.295323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf20c95-5e62-44b6-94ac-44d3a76ef691 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:07.296542 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-896d5cef-fe09-4d15-b2b6-308dd2bd8f20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b82af73e810a4053a0a125077f1b22b5', 'request_id': 'req-896d5cef-fe09-4d15-b2b6-308dd2bd8f20'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:07.305881 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-896d5cef-fe09-4d15-b2b6-308dd2bd8f20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b82af73e810a4053a0a125077f1b22b5', 'request_id': 'req-896d5cef-fe09-4d15-b2b6-308dd2bd8f20'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:07.307137 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:07.318355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-896d5cef-fe09-4d15-b2b6-308dd2bd8f20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b82af73e810a4053a0a125077f1b22b5', 'request_id': 'req-896d5cef-fe09-4d15-b2b6-308dd2bd8f20'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:07.330974 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-896d5cef-fe09-4d15-b2b6-308dd2bd8f20 admin admin] The token cache is being invalidate because project b82af73e810a4053a0a125077f1b22b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:31:07.331266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-896d5cef-fe09-4d15-b2b6-308dd2bd8f20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:07.331975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-896d5cef-fe09-4d15-b2b6-308dd2bd8f20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:07.334468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1275/2544] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:31:07 2022] DELETE /identity/v3/projects/b82af73e810a4053a0a125077f1b22b5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:07.361439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b7682-3d44-4a06-9c79-64bcf495c69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:07.362221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b7682-3d44-4a06-9c79-64bcf495c69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:07.362565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b7682-3d44-4a06-9c79-64bcf495c69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:07.362908 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b7682-3d44-4a06-9c79-64bcf495c69c admin admin] PATH_INFO: `/v3/users/149a82ddc13c4e73a58d93d9f238ace9` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:07.364252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b7682-3d44-4a06-9c79-64bcf495c69c admin admin] RBAC: Authorizing `identity:delete_user(user_id=149a82ddc13c4e73a58d93d9f238ace9)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:07.366238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b7682-3d44-4a06-9c79-64bcf495c69c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:07.369604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:07.428150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d14b7682-3d44-4a06-9c79-64bcf495c69c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '149a82ddc13c4e73a58d93d9f238ace9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:07.436027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d14b7682-3d44-4a06-9c79-64bcf495c69c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '149a82ddc13c4e73a58d93d9f238ace9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:07.436822 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d14b7682-3d44-4a06-9c79-64bcf495c69c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '149a82ddc13c4e73a58d93d9f238ace9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:07.437430 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d14b7682-3d44-4a06-9c79-64bcf495c69c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '149a82ddc13c4e73a58d93d9f238ace9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:07.446759 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-d14b7682-3d44-4a06-9c79-64bcf495c69c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '149a82ddc13c4e73a58d93d9f238ace9'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:07.466076 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1270/2545] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:31:07 2022] DELETE /identity/v3/users/149a82ddc13c4e73a58d93d9f238ace9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:31:07.470762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee667a62-520a-4498-a61b-b81c82dc725b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:07.471148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee667a62-520a-4498-a61b-b81c82dc725b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:07.471538 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee667a62-520a-4498-a61b-b81c82dc725b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:07.492079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ee667a62-520a-4498-a61b-b81c82dc725b None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:07.493415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ee667a62-520a-4498-a61b-b81c82dc725b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:07.557357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1276/2546] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:07.567231 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b7682-3d44-4a06-9c79-64bcf495c69c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:07.568798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d14b7682-3d44-4a06-9c79-64bcf495c69c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:07.627022 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b7682-3d44-4a06-9c79-64bcf495c69c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:07.630677 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70aaab5-23fd-4dc2-9382-775e73390127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:07.631289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70aaab5-23fd-4dc2-9382-775e73390127 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:07.631583 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70aaab5-23fd-4dc2-9382-775e73390127 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:07.631857 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70aaab5-23fd-4dc2-9382-775e73390127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:07.640707 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70aaab5-23fd-4dc2-9382-775e73390127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:07.642109 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70aaab5-23fd-4dc2-9382-775e73390127 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:07.652098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1271/2547] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:31:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:31:07.916376 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee667a62-520a-4498-a61b-b81c82dc725b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:07.917896 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ee667a62-520a-4498-a61b-b81c82dc725b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:07.930690 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee667a62-520a-4498-a61b-b81c82dc725b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:07.935109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:07.935831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:07.936110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:07.936383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899 admin admin] PATH_INFO: `/v3/projects/b9554a6be96c423bb5dd5125c2920a61` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:07.937504 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9554a6be96c423bb5dd5125c2920a61)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:07.939389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:07.986732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9554a6be96c423bb5dd5125c2920a61', 'request_id': 'req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:07.994602 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9554a6be96c423bb5dd5125c2920a61', 'request_id': 'req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:08.000623 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9554a6be96c423bb5dd5125c2920a61', 'request_id': 'req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:08.011160 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899 admin admin] The token cache is being invalidate because project b9554a6be96c423bb5dd5125c2920a61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:31:08.011448 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:08.012097 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:08.013563 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1277/2548] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:31:07 2022] DELETE /identity/v3/projects/b9554a6be96c423bb5dd5125c2920a61 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:08.017539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70aaab5-23fd-4dc2-9382-775e73390127 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:08.027765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70aaab5-23fd-4dc2-9382-775e73390127 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:08.082232 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431103e7-62aa-4f84-a77a-0226852bab32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:08.082881 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431103e7-62aa-4f84-a77a-0226852bab32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:08.083287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431103e7-62aa-4f84-a77a-0226852bab32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:08.083524 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431103e7-62aa-4f84-a77a-0226852bab32 admin admin] PATH_INFO: `/v3/users/addd018c429643b3aaed4bb1aa4de9fa` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:08.084757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431103e7-62aa-4f84-a77a-0226852bab32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=addd018c429643b3aaed4bb1aa4de9fa)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:08.087280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431103e7-62aa-4f84-a77a-0226852bab32 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:08.114083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:08.135114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-431103e7-62aa-4f84-a77a-0226852bab32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'addd018c429643b3aaed4bb1aa4de9fa'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:08.142039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-431103e7-62aa-4f84-a77a-0226852bab32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'addd018c429643b3aaed4bb1aa4de9fa'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:08.142715 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-431103e7-62aa-4f84-a77a-0226852bab32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'addd018c429643b3aaed4bb1aa4de9fa'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:08.143323 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-431103e7-62aa-4f84-a77a-0226852bab32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'addd018c429643b3aaed4bb1aa4de9fa'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:08.150757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-431103e7-62aa-4f84-a77a-0226852bab32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'addd018c429643b3aaed4bb1aa4de9fa'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:08.167717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1272/2549] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:31:08 2022] DELETE /identity/v3/users/addd018c429643b3aaed4bb1aa4de9fa => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:08.449780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:08.451233 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:08.468214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1bf95-c791-4bdf-9b44-a9bd1cdaa899 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:08.525988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8c9e7-4117-4d9b-a93a-a8ba68b3778e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:08.526535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8c9e7-4117-4d9b-a93a-a8ba68b3778e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:08.526820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8c9e7-4117-4d9b-a93a-a8ba68b3778e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:08.527100 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8c9e7-4117-4d9b-a93a-a8ba68b3778e admin admin] PATH_INFO: `/v3/projects/4696979293fc426bb08adbb6162cdfa8` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:08.528259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8c9e7-4117-4d9b-a93a-a8ba68b3778e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4696979293fc426bb08adbb6162cdfa8)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:08.530153 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8c9e7-4117-4d9b-a93a-a8ba68b3778e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:08.541040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:08.579011 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-18f8c9e7-4117-4d9b-a93a-a8ba68b3778e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4696979293fc426bb08adbb6162cdfa8', 'request_id': 'req-18f8c9e7-4117-4d9b-a93a-a8ba68b3778e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:08.586846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-18f8c9e7-4117-4d9b-a93a-a8ba68b3778e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4696979293fc426bb08adbb6162cdfa8', 'request_id': 'req-18f8c9e7-4117-4d9b-a93a-a8ba68b3778e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:08.593498 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-18f8c9e7-4117-4d9b-a93a-a8ba68b3778e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4696979293fc426bb08adbb6162cdfa8', 'request_id': 'req-18f8c9e7-4117-4d9b-a93a-a8ba68b3778e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:08.604361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-18f8c9e7-4117-4d9b-a93a-a8ba68b3778e admin admin] The token cache is being invalidate because project 4696979293fc426bb08adbb6162cdfa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:31:08.604649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-18f8c9e7-4117-4d9b-a93a-a8ba68b3778e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:08.605298 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-18f8c9e7-4117-4d9b-a93a-a8ba68b3778e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:08.606804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1278/2550] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:31:08 2022] DELETE /identity/v3/projects/4696979293fc426bb08adbb6162cdfa8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:08.612018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238c025-7aaf-4ddb-98ff-4836d1b2e7a7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:08.612544 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238c025-7aaf-4ddb-98ff-4836d1b2e7a7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:08.612544 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238c025-7aaf-4ddb-98ff-4836d1b2e7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:08.632542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-2238c025-7aaf-4ddb-98ff-4836d1b2e7a7 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:08.633783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2238c025-7aaf-4ddb-98ff-4836d1b2e7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:08.691801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1273/2551] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:08.699759 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e626f2-e068-4ea5-88ad-a8c6189ff0f2 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:08.700038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e626f2-e068-4ea5-88ad-a8c6189ff0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:08.700315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e626f2-e068-4ea5-88ad-a8c6189ff0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:08.721529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-70e626f2-e068-4ea5-88ad-a8c6189ff0f2 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:08.722749 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-70e626f2-e068-4ea5-88ad-a8c6189ff0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:08.731523 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1279/2552] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:08.736082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238c025-7aaf-4ddb-98ff-4836d1b2e7a7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:08.746536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238c025-7aaf-4ddb-98ff-4836d1b2e7a7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:08.751125 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f69d57-7d62-4ffc-a3c5-98100f2e91b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:08.751761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f69d57-7d62-4ffc-a3c5-98100f2e91b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:08.752043 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f69d57-7d62-4ffc-a3c5-98100f2e91b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:08.752316 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f69d57-7d62-4ffc-a3c5-98100f2e91b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:08.752956 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f69d57-7d62-4ffc-a3c5-98100f2e91b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:31:08.753256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f69d57-7d62-4ffc-a3c5-98100f2e91b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:08.754844 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f69d57-7d62-4ffc-a3c5-98100f2e91b2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:08.766245 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1274/2553] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:31:08.771599 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e626f2-e068-4ea5-88ad-a8c6189ff0f2 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:08.780947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e626f2-e068-4ea5-88ad-a8c6189ff0f2 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:08.785577 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37112b94-2983-40d1-9984-3db3d49dbb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:08.786172 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37112b94-2983-40d1-9984-3db3d49dbb35 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:08.786456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37112b94-2983-40d1-9984-3db3d49dbb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:08.786730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37112b94-2983-40d1-9984-3db3d49dbb35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:08.787365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37112b94-2983-40d1-9984-3db3d49dbb35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:08.789877 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37112b94-2983-40d1-9984-3db3d49dbb35 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:08.816951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1280/2554] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:31:08 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:31:08.821200 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f69d57-7d62-4ffc-a3c5-98100f2e91b2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:08.831253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f69d57-7d62-4ffc-a3c5-98100f2e91b2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:08.888113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a09d18-959c-412a-89ab-a146f8285fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:08.889040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a09d18-959c-412a-89ab-a146f8285fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:08.889326 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a09d18-959c-412a-89ab-a146f8285fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:08.889596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a09d18-959c-412a-89ab-a146f8285fd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:08.890308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a09d18-959c-412a-89ab-a146f8285fd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:08.892235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a09d18-959c-412a-89ab-a146f8285fd0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:08.915138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1275/2555] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:31:08 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:31:08.919387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37112b94-2983-40d1-9984-3db3d49dbb35 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:08.929747 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37112b94-2983-40d1-9984-3db3d49dbb35 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:08.934442 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd34ce4-2ff5-4f6a-93f5-2fa00c36fe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:08.935042 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd34ce4-2ff5-4f6a-93f5-2fa00c36fe8e admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:08.935327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd34ce4-2ff5-4f6a-93f5-2fa00c36fe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:08.935643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd34ce4-2ff5-4f6a-93f5-2fa00c36fe8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:08.936294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd34ce4-2ff5-4f6a-93f5-2fa00c36fe8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:08.938018 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd34ce4-2ff5-4f6a-93f5-2fa00c36fe8e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:08.948342 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1281/2556] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:31:08.952537 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a09d18-959c-412a-89ab-a146f8285fd0 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:08.964722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a09d18-959c-412a-89ab-a146f8285fd0 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:08.969732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f329c7-c800-416f-8039-28f1b9b1f17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:08.970553 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f329c7-c800-416f-8039-28f1b9b1f17e admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:08.970869 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f329c7-c800-416f-8039-28f1b9b1f17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:08.971145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f329c7-c800-416f-8039-28f1b9b1f17e admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:08.972286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f329c7-c800-416f-8039-28f1b9b1f17e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:08.974187 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f329c7-c800-416f-8039-28f1b9b1f17e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:08.986615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1276/2557] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:31:08.990900 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd34ce4-2ff5-4f6a-93f5-2fa00c36fe8e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:09.001598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd34ce4-2ff5-4f6a-93f5-2fa00c36fe8e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:09.006294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d10ad-39c3-4527-95e5-41f1838df9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:09.006941 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d10ad-39c3-4527-95e5-41f1838df9fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:09.007253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d10ad-39c3-4527-95e5-41f1838df9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:09.007610 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d10ad-39c3-4527-95e5-41f1838df9fa admin admin] PATH_INFO: `/v3/projects/3542ce70dad74ce1b6e482a10e48c983/users/a9a6bcf5b46e4ce7a9b05484b3838c30/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:09.024296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d10ad-39c3-4527-95e5-41f1838df9fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3542ce70dad74ce1b6e482a10e48c983, user_id=a9a6bcf5b46e4ce7a9b05484b3838c30, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:09.026997 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d10ad-39c3-4527-95e5-41f1838df9fa admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:09.040001 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1282/2558] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:31:08 2022] PUT /identity/v3/projects/3542ce70dad74ce1b6e482a10e48c983/users/a9a6bcf5b46e4ce7a9b05484b3838c30/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:09.050138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c269583-6232-484b-901b-1e9beadfe793 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:09.050420 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c269583-6232-484b-901b-1e9beadfe793 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:09.050693 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c269583-6232-484b-901b-1e9beadfe793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:09.071069 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:09.085625 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-0c269583-6232-484b-901b-1e9beadfe793 None None] MFA Rules not processed for user `a9a6bcf5b46e4ce7a9b05484b3838c30`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:09.086797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0c269583-6232-484b-901b-1e9beadfe793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:09.150721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1277/2559] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:31:09.155363 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d10ad-39c3-4527-95e5-41f1838df9fa admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:09.215897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d10ad-39c3-4527-95e5-41f1838df9fa admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:09.220524 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd2f2e-8357-48ac-9e2a-eef100d61aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:09.221123 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd2f2e-8357-48ac-9e2a-eef100d61aac admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:09.221406 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd2f2e-8357-48ac-9e2a-eef100d61aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:09.221680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd2f2e-8357-48ac-9e2a-eef100d61aac admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:09.222314 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd2f2e-8357-48ac-9e2a-eef100d61aac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:09.224240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd2f2e-8357-48ac-9e2a-eef100d61aac admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:09.252415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1283/2560] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:31:09 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:31:09.256574 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c269583-6232-484b-901b-1e9beadfe793 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:09.267164 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c269583-6232-484b-901b-1e9beadfe793 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:09.323181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df30422-68d6-49bd-923d-efe4cc291bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:09.323721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df30422-68d6-49bd-923d-efe4cc291bc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:09.324002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df30422-68d6-49bd-923d-efe4cc291bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:09.324275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df30422-68d6-49bd-923d-efe4cc291bc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:09.324896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df30422-68d6-49bd-923d-efe4cc291bc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:09.326789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df30422-68d6-49bd-923d-efe4cc291bc8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:09.347357 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1278/2561] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:31:09 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:31:09.352010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd2f2e-8357-48ac-9e2a-eef100d61aac admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:09.362104 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd2f2e-8357-48ac-9e2a-eef100d61aac admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:09.366728 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916070f6-374b-455b-b035-8f9840051908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:09.367833 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916070f6-374b-455b-b035-8f9840051908 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:09.368240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916070f6-374b-455b-b035-8f9840051908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:09.368513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916070f6-374b-455b-b035-8f9840051908 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:09.369140 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916070f6-374b-455b-b035-8f9840051908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:09.370732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916070f6-374b-455b-b035-8f9840051908 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:09.380990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1284/2562] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:31:09.385104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df30422-68d6-49bd-923d-efe4cc291bc8 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:09.395283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df30422-68d6-49bd-923d-efe4cc291bc8 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:09.399963 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3305b2e-b450-4e17-931c-b1056d406109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:09.400598 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3305b2e-b450-4e17-931c-b1056d406109 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:09.400874 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3305b2e-b450-4e17-931c-b1056d406109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:09.401143 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3305b2e-b450-4e17-931c-b1056d406109 admin admin] PATH_INFO: `/v3/projects/88f68695a3954958b6a302e7c96c833e/users/2e3ecb71719942f0bdda08eaacf29be0/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:09.416173 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3305b2e-b450-4e17-931c-b1056d406109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f68695a3954958b6a302e7c96c833e, user_id=2e3ecb71719942f0bdda08eaacf29be0, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:09.418988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3305b2e-b450-4e17-931c-b1056d406109 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:09.432569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1279/2563] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:31:09 2022] PUT /identity/v3/projects/88f68695a3954958b6a302e7c96c833e/users/2e3ecb71719942f0bdda08eaacf29be0/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:09.436518 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916070f6-374b-455b-b035-8f9840051908 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:09.446344 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916070f6-374b-455b-b035-8f9840051908 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:09.450875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:09.504124 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c136a3-d6fe-467c-940d-39035bf62c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:09.504727 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c136a3-d6fe-467c-940d-39035bf62c71 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:09.505007 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c136a3-d6fe-467c-940d-39035bf62c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:09.505278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c136a3-d6fe-467c-940d-39035bf62c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:09.505934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c136a3-d6fe-467c-940d-39035bf62c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:09.507575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c136a3-d6fe-467c-940d-39035bf62c71 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:09.517100 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1285/2564] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:31:09.521647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3305b2e-b450-4e17-931c-b1056d406109 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:09.531616 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3305b2e-b450-4e17-931c-b1056d406109 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:09.536097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197706cb-b9ff-46a2-990f-914cdc7b25b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:09.536736 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197706cb-b9ff-46a2-990f-914cdc7b25b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:09.537016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197706cb-b9ff-46a2-990f-914cdc7b25b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:09.537287 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197706cb-b9ff-46a2-990f-914cdc7b25b9 admin admin] PATH_INFO: `/v3/projects/88f68695a3954958b6a302e7c96c833e/users/2e3ecb71719942f0bdda08eaacf29be0/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:09.539758 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197706cb-b9ff-46a2-990f-914cdc7b25b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f68695a3954958b6a302e7c96c833e, user_id=2e3ecb71719942f0bdda08eaacf29be0, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:09.542348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197706cb-b9ff-46a2-990f-914cdc7b25b9 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:09.552273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1280/2565] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:31:09 2022] PUT /identity/v3/projects/88f68695a3954958b6a302e7c96c833e/users/2e3ecb71719942f0bdda08eaacf29be0/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:09.617765 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed89e6e-5a86-46e8-bca6-a961aabf057d None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:09.618049 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed89e6e-5a86-46e8-bca6-a961aabf057d None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:09.618321 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed89e6e-5a86-46e8-bca6-a961aabf057d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:09.645551 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-0ed89e6e-5a86-46e8-bca6-a961aabf057d None None] MFA Rules not processed for user `2e3ecb71719942f0bdda08eaacf29be0`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:09.646741 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0ed89e6e-5a86-46e8-bca6-a961aabf057d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:09.712950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1286/2566] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:09.723527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197706cb-b9ff-46a2-990f-914cdc7b25b9 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:09.725327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-197706cb-b9ff-46a2-990f-914cdc7b25b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:09.781860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197706cb-b9ff-46a2-990f-914cdc7b25b9 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:09.785518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94284d7c-2326-43b8-8a64-5e4b77d82685 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:09.786112 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94284d7c-2326-43b8-8a64-5e4b77d82685 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:09.786393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94284d7c-2326-43b8-8a64-5e4b77d82685 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:09.786664 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94284d7c-2326-43b8-8a64-5e4b77d82685 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:09.793948 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:09.798436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94284d7c-2326-43b8-8a64-5e4b77d82685 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:09.800501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94284d7c-2326-43b8-8a64-5e4b77d82685 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:09.810128 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1281/2567] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:31:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:31:10.177565 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed89e6e-5a86-46e8-bca6-a961aabf057d None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:10.236494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed89e6e-5a86-46e8-bca6-a961aabf057d None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:10.240975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933bed4-9b3e-4557-a678-c789dd6e5088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:10.241625 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933bed4-9b3e-4557-a678-c789dd6e5088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:10.242421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933bed4-9b3e-4557-a678-c789dd6e5088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:10.242685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933bed4-9b3e-4557-a678-c789dd6e5088 admin admin] PATH_INFO: `/v3/users/a9a6bcf5b46e4ce7a9b05484b3838c30` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:10.243820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c933bed4-9b3e-4557-a678-c789dd6e5088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9a6bcf5b46e4ce7a9b05484b3838c30)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:10.245725 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c933bed4-9b3e-4557-a678-c789dd6e5088 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:10.278844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:10.291849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c933bed4-9b3e-4557-a678-c789dd6e5088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9a6bcf5b46e4ce7a9b05484b3838c30'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:10.298679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c933bed4-9b3e-4557-a678-c789dd6e5088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9a6bcf5b46e4ce7a9b05484b3838c30'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:10.299378 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c933bed4-9b3e-4557-a678-c789dd6e5088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9a6bcf5b46e4ce7a9b05484b3838c30'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:10.300023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c933bed4-9b3e-4557-a678-c789dd6e5088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9a6bcf5b46e4ce7a9b05484b3838c30'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:10.307694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-c933bed4-9b3e-4557-a678-c789dd6e5088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9a6bcf5b46e4ce7a9b05484b3838c30'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:10.325133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1287/2568] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:31:10 2022] DELETE /identity/v3/users/a9a6bcf5b46e4ce7a9b05484b3838c30 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:10.329658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39ebc4-87eb-4d78-8939-39c9243cc8e7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:10.329944 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39ebc4-87eb-4d78-8939-39c9243cc8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:10.330216 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39ebc4-87eb-4d78-8939-39c9243cc8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:10.349069 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fc39ebc4-87eb-4d78-8939-39c9243cc8e7 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:10.350272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fc39ebc4-87eb-4d78-8939-39c9243cc8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:10.407955 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1282/2569] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:10.417729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933bed4-9b3e-4557-a678-c789dd6e5088 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:10.419110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c933bed4-9b3e-4557-a678-c789dd6e5088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:10.474059 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933bed4-9b3e-4557-a678-c789dd6e5088 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:10.477649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f7170-d9dc-48da-aa60-5e92f34d1296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:10.478236 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f7170-d9dc-48da-aa60-5e92f34d1296 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:10.478511 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f7170-d9dc-48da-aa60-5e92f34d1296 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:10.478772 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f7170-d9dc-48da-aa60-5e92f34d1296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:10.488189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f7170-d9dc-48da-aa60-5e92f34d1296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:10.489799 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f7170-d9dc-48da-aa60-5e92f34d1296 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:10.500270 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1288/2570] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:31:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:31:10.768347 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39ebc4-87eb-4d78-8939-39c9243cc8e7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:10.770273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fc39ebc4-87eb-4d78-8939-39c9243cc8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:10.782536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39ebc4-87eb-4d78-8939-39c9243cc8e7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:10.787169 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62779d-9de1-4d95-9bef-e763d90e4f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:10.787837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62779d-9de1-4d95-9bef-e763d90e4f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:10.788119 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62779d-9de1-4d95-9bef-e763d90e4f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:10.788920 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62779d-9de1-4d95-9bef-e763d90e4f80 admin admin] PATH_INFO: `/v3/projects/3542ce70dad74ce1b6e482a10e48c983` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:10.790163 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62779d-9de1-4d95-9bef-e763d90e4f80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3542ce70dad74ce1b6e482a10e48c983)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:10.792208 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62779d-9de1-4d95-9bef-e763d90e4f80 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:10.838931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1d62779d-9de1-4d95-9bef-e763d90e4f80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3542ce70dad74ce1b6e482a10e48c983', 'request_id': 'req-1d62779d-9de1-4d95-9bef-e763d90e4f80'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:10.846894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1d62779d-9de1-4d95-9bef-e763d90e4f80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3542ce70dad74ce1b6e482a10e48c983', 'request_id': 'req-1d62779d-9de1-4d95-9bef-e763d90e4f80'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:10.853646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1d62779d-9de1-4d95-9bef-e763d90e4f80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3542ce70dad74ce1b6e482a10e48c983', 'request_id': 'req-1d62779d-9de1-4d95-9bef-e763d90e4f80'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:10.864075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1d62779d-9de1-4d95-9bef-e763d90e4f80 admin admin] The token cache is being invalidate because project 3542ce70dad74ce1b6e482a10e48c983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:31:10.864366 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1d62779d-9de1-4d95-9bef-e763d90e4f80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:10.865006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1d62779d-9de1-4d95-9bef-e763d90e4f80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:10.866507 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1283/2571] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:31:10 2022] DELETE /identity/v3/projects/3542ce70dad74ce1b6e482a10e48c983 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:10.871005 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f7170-d9dc-48da-aa60-5e92f34d1296 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:10.880616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f7170-d9dc-48da-aa60-5e92f34d1296 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:10.937024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d0ce9-111a-44fa-a63b-6eb148535938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:10.937651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d0ce9-111a-44fa-a63b-6eb148535938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:10.937931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d0ce9-111a-44fa-a63b-6eb148535938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:10.938199 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d0ce9-111a-44fa-a63b-6eb148535938 admin admin] PATH_INFO: `/v3/users/2e3ecb71719942f0bdda08eaacf29be0` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:10.939304 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2d0ce9-111a-44fa-a63b-6eb148535938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e3ecb71719942f0bdda08eaacf29be0)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:10.941275 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2d0ce9-111a-44fa-a63b-6eb148535938 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:10.984319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cf2d0ce9-111a-44fa-a63b-6eb148535938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e3ecb71719942f0bdda08eaacf29be0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:10.989390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:10.991284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cf2d0ce9-111a-44fa-a63b-6eb148535938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e3ecb71719942f0bdda08eaacf29be0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:10.992799 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cf2d0ce9-111a-44fa-a63b-6eb148535938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e3ecb71719942f0bdda08eaacf29be0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:10.993695 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cf2d0ce9-111a-44fa-a63b-6eb148535938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e3ecb71719942f0bdda08eaacf29be0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:11.001393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-cf2d0ce9-111a-44fa-a63b-6eb148535938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e3ecb71719942f0bdda08eaacf29be0'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:11.018225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1289/2572] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:31:10 2022] DELETE /identity/v3/users/2e3ecb71719942f0bdda08eaacf29be0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:11.283789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62779d-9de1-4d95-9bef-e763d90e4f80 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:11.285830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1d62779d-9de1-4d95-9bef-e763d90e4f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:11.301854 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62779d-9de1-4d95-9bef-e763d90e4f80 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:11.359017 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfcaf9-ba17-4959-86f2-ad671cf7500a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:11.359730 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfcaf9-ba17-4959-86f2-ad671cf7500a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:11.360008 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfcaf9-ba17-4959-86f2-ad671cf7500a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:11.360276 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfcaf9-ba17-4959-86f2-ad671cf7500a admin admin] PATH_INFO: `/v3/projects/88f68695a3954958b6a302e7c96c833e` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:11.361374 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbfcaf9-ba17-4959-86f2-ad671cf7500a admin admin] RBAC: Authorizing `identity:delete_project(project_id=88f68695a3954958b6a302e7c96c833e)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:11.363286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbfcaf9-ba17-4959-86f2-ad671cf7500a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:11.389751 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:11.417962 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4cbfcaf9-ba17-4959-86f2-ad671cf7500a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88f68695a3954958b6a302e7c96c833e', 'request_id': 'req-4cbfcaf9-ba17-4959-86f2-ad671cf7500a'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:11.431701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4cbfcaf9-ba17-4959-86f2-ad671cf7500a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88f68695a3954958b6a302e7c96c833e', 'request_id': 'req-4cbfcaf9-ba17-4959-86f2-ad671cf7500a'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:11.446105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4cbfcaf9-ba17-4959-86f2-ad671cf7500a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88f68695a3954958b6a302e7c96c833e', 'request_id': 'req-4cbfcaf9-ba17-4959-86f2-ad671cf7500a'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:11.460825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4cbfcaf9-ba17-4959-86f2-ad671cf7500a admin admin] The token cache is being invalidate because project 88f68695a3954958b6a302e7c96c833e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:31:11.461348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4cbfcaf9-ba17-4959-86f2-ad671cf7500a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:11.462419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-4cbfcaf9-ba17-4959-86f2-ad671cf7500a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:11.465423 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1284/2573] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:31:11 2022] DELETE /identity/v3/projects/88f68695a3954958b6a302e7c96c833e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:11.469942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75680122-9f2a-44cb-a6f0-075ead348248 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:11.470466 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75680122-9f2a-44cb-a6f0-075ead348248 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:11.470600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75680122-9f2a-44cb-a6f0-075ead348248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:11.494475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-75680122-9f2a-44cb-a6f0-075ead348248 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:11.495748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-75680122-9f2a-44cb-a6f0-075ead348248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:11.555988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1290/2574] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:11.563611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5ed32-2fb1-49b6-9cec-f1241f4a28a7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:11.564114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5ed32-2fb1-49b6-9cec-f1241f4a28a7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:11.564317 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5ed32-2fb1-49b6-9cec-f1241f4a28a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:11.586116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-40c5ed32-2fb1-49b6-9cec-f1241f4a28a7 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:11.587303 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-40c5ed32-2fb1-49b6-9cec-f1241f4a28a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:11.595119 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1285/2575] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:11.599380 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75680122-9f2a-44cb-a6f0-075ead348248 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:11.608735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75680122-9f2a-44cb-a6f0-075ead348248 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:11.613198 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b06d7c-6812-4942-8373-df4a9b90d7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:11.613786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b06d7c-6812-4942-8373-df4a9b90d7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:11.614064 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b06d7c-6812-4942-8373-df4a9b90d7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:11.614332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b06d7c-6812-4942-8373-df4a9b90d7a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:11.614969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b06d7c-6812-4942-8373-df4a9b90d7a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:31:11.615269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b06d7c-6812-4942-8373-df4a9b90d7a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:11.616953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b06d7c-6812-4942-8373-df4a9b90d7a2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:11.628176 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1291/2576] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:31:11.632888 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5ed32-2fb1-49b6-9cec-f1241f4a28a7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:11.641995 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5ed32-2fb1-49b6-9cec-f1241f4a28a7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:11.646854 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9ba32-1585-4c6d-87fb-12f74d4bae42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:11.647442 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9ba32-1585-4c6d-87fb-12f74d4bae42 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:11.648365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9ba32-1585-4c6d-87fb-12f74d4bae42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:11.648673 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9ba32-1585-4c6d-87fb-12f74d4bae42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:11.649291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9ba32-1585-4c6d-87fb-12f74d4bae42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:11.651167 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9ba32-1585-4c6d-87fb-12f74d4bae42 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:11.677661 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1286/2577] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:31:11 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:31:11.682084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b06d7c-6812-4942-8373-df4a9b90d7a2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:11.691762 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b06d7c-6812-4942-8373-df4a9b90d7a2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:11.745355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ef221-baf7-4e31-af3f-330177cde13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:11.745992 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ef221-baf7-4e31-af3f-330177cde13a admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:11.746216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ef221-baf7-4e31-af3f-330177cde13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:11.746484 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ef221-baf7-4e31-af3f-330177cde13a admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:11.747091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ef221-baf7-4e31-af3f-330177cde13a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:11.749000 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ef221-baf7-4e31-af3f-330177cde13a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:11.771657 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1292/2578] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:31:11 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:31:11.776370 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9ba32-1585-4c6d-87fb-12f74d4bae42 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:11.786619 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9ba32-1585-4c6d-87fb-12f74d4bae42 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:11.791199 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ad265-df7e-4baa-a1fb-b07a498a41f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:11.791824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ad265-df7e-4baa-a1fb-b07a498a41f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:11.792105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ad265-df7e-4baa-a1fb-b07a498a41f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:11.792379 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ad265-df7e-4baa-a1fb-b07a498a41f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:11.793013 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ad265-df7e-4baa-a1fb-b07a498a41f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:11.794585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ad265-df7e-4baa-a1fb-b07a498a41f0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:11.804231 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1287/2579] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:31:11.808135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ef221-baf7-4e31-af3f-330177cde13a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:11.818407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ef221-baf7-4e31-af3f-330177cde13a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:11.823524 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085ff10-54e6-4d25-ad63-e7fadf6e0846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:11.824509 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085ff10-54e6-4d25-ad63-e7fadf6e0846 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:11.824790 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085ff10-54e6-4d25-ad63-e7fadf6e0846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:11.825058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085ff10-54e6-4d25-ad63-e7fadf6e0846 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:11.825694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085ff10-54e6-4d25-ad63-e7fadf6e0846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:11.827278 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085ff10-54e6-4d25-ad63-e7fadf6e0846 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:11.838277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1293/2580] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:31:11.842346 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ad265-df7e-4baa-a1fb-b07a498a41f0 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:11.852218 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ad265-df7e-4baa-a1fb-b07a498a41f0 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:11.856774 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a44ed-2142-439e-8a46-dad57710416a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:11.857412 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a44ed-2142-439e-8a46-dad57710416a admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:11.857690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a44ed-2142-439e-8a46-dad57710416a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:11.857959 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a44ed-2142-439e-8a46-dad57710416a admin admin] PATH_INFO: `/v3/projects/0a24adabd6a24b7bb5ddd05ef059971f/users/de89c1424e16461c9e3941857c0d466b/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:11.872531 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a44ed-2142-439e-8a46-dad57710416a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a24adabd6a24b7bb5ddd05ef059971f, user_id=de89c1424e16461c9e3941857c0d466b, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:11.875149 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a44ed-2142-439e-8a46-dad57710416a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:11.888211 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1288/2581] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:31:11 2022] PUT /identity/v3/projects/0a24adabd6a24b7bb5ddd05ef059971f/users/de89c1424e16461c9e3941857c0d466b/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:11.898333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e73a4-1c3b-44f1-bfe7-2abbdbb7a835 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:11.898621 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e73a4-1c3b-44f1-bfe7-2abbdbb7a835 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:11.898894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e73a4-1c3b-44f1-bfe7-2abbdbb7a835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:11.925837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ed8e73a4-1c3b-44f1-bfe7-2abbdbb7a835 None None] MFA Rules not processed for user `de89c1424e16461c9e3941857c0d466b`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:11.927033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ed8e73a4-1c3b-44f1-bfe7-2abbdbb7a835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:11.953284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:11.992641 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1294/2582] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:12.003642 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a44ed-2142-439e-8a46-dad57710416a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:12.005327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ed4a44ed-2142-439e-8a46-dad57710416a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:12.060593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a44ed-2142-439e-8a46-dad57710416a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:12.064198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3bd6fb-2130-41af-b8a9-097942cc375c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:12.064788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bd6fb-2130-41af-b8a9-097942cc375c service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:12.065068 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bd6fb-2130-41af-b8a9-097942cc375c service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:12.065340 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bd6fb-2130-41af-b8a9-097942cc375c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:12.074045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3bd6fb-2130-41af-b8a9-097942cc375c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:12.075452 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3bd6fb-2130-41af-b8a9-097942cc375c service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:12.084078 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1289/2583] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:31:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:31:12.104159 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e73a4-1c3b-44f1-bfe7-2abbdbb7a835 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:12.164611 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e73a4-1c3b-44f1-bfe7-2abbdbb7a835 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:12.169106 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fc72d-5724-4c79-9cfe-b43ba907dea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:12.169716 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fc72d-5724-4c79-9cfe-b43ba907dea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:12.169991 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fc72d-5724-4c79-9cfe-b43ba907dea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:12.170259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fc72d-5724-4c79-9cfe-b43ba907dea8 admin admin] PATH_INFO: `/v3/users/de89c1424e16461c9e3941857c0d466b` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:12.171425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fc72d-5724-4c79-9cfe-b43ba907dea8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de89c1424e16461c9e3941857c0d466b)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:12.173349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fc72d-5724-4c79-9cfe-b43ba907dea8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:12.218354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-785fc72d-5724-4c79-9cfe-b43ba907dea8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de89c1424e16461c9e3941857c0d466b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:12.225155 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-785fc72d-5724-4c79-9cfe-b43ba907dea8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de89c1424e16461c9e3941857c0d466b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:12.226376 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-785fc72d-5724-4c79-9cfe-b43ba907dea8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de89c1424e16461c9e3941857c0d466b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:12.227001 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-785fc72d-5724-4c79-9cfe-b43ba907dea8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de89c1424e16461c9e3941857c0d466b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:12.234700 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-785fc72d-5724-4c79-9cfe-b43ba907dea8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de89c1424e16461c9e3941857c0d466b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:12.252648 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1295/2584] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:31:12 2022] DELETE /identity/v3/users/de89c1424e16461c9e3941857c0d466b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:12.257097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8b14b-ffde-48be-a2f4-9df7fdc87282 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:12.257381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8b14b-ffde-48be-a2f4-9df7fdc87282 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:12.257650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8b14b-ffde-48be-a2f4-9df7fdc87282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:12.276356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-15d8b14b-ffde-48be-a2f4-9df7fdc87282 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:12.277531 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-15d8b14b-ffde-48be-a2f4-9df7fdc87282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:12.317746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:12.335330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1290/2585] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:12.345080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fc72d-5724-4c79-9cfe-b43ba907dea8 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:12.346480 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-785fc72d-5724-4c79-9cfe-b43ba907dea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:12.397885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:12.404147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fc72d-5724-4c79-9cfe-b43ba907dea8 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:12.408302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bc582-6a50-4ce7-8db9-f27e8f7b3e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:12.408920 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bc582-6a50-4ce7-8db9-f27e8f7b3e58 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:12.409316 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bc582-6a50-4ce7-8db9-f27e8f7b3e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:12.409645 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bc582-6a50-4ce7-8db9-f27e8f7b3e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:12.419196 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bc582-6a50-4ce7-8db9-f27e8f7b3e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:12.420653 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bc582-6a50-4ce7-8db9-f27e8f7b3e58 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:12.430323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1296/2586] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:31:12.690195 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8b14b-ffde-48be-a2f4-9df7fdc87282 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:12.691809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-15d8b14b-ffde-48be-a2f4-9df7fdc87282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:12.710826 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8b14b-ffde-48be-a2f4-9df7fdc87282 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:12.716770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed50f4-6d69-4374-99a5-f173d5c0a936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:12.717421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed50f4-6d69-4374-99a5-f173d5c0a936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:12.717707 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed50f4-6d69-4374-99a5-f173d5c0a936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:12.717985 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed50f4-6d69-4374-99a5-f173d5c0a936 admin admin] PATH_INFO: `/v3/projects/0a24adabd6a24b7bb5ddd05ef059971f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:12.719226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed50f4-6d69-4374-99a5-f173d5c0a936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a24adabd6a24b7bb5ddd05ef059971f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:12.721209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed50f4-6d69-4374-99a5-f173d5c0a936 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:12.768394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-62ed50f4-6d69-4374-99a5-f173d5c0a936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a24adabd6a24b7bb5ddd05ef059971f', 'request_id': 'req-62ed50f4-6d69-4374-99a5-f173d5c0a936'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:12.776321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-62ed50f4-6d69-4374-99a5-f173d5c0a936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a24adabd6a24b7bb5ddd05ef059971f', 'request_id': 'req-62ed50f4-6d69-4374-99a5-f173d5c0a936'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:12.782602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-62ed50f4-6d69-4374-99a5-f173d5c0a936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a24adabd6a24b7bb5ddd05ef059971f', 'request_id': 'req-62ed50f4-6d69-4374-99a5-f173d5c0a936'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:12.792986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-62ed50f4-6d69-4374-99a5-f173d5c0a936 admin admin] The token cache is being invalidate because project 0a24adabd6a24b7bb5ddd05ef059971f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:31:12.793274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-62ed50f4-6d69-4374-99a5-f173d5c0a936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:12.793915 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-62ed50f4-6d69-4374-99a5-f173d5c0a936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:12.795389 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1291/2587] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:31:12 2022] DELETE /identity/v3/projects/0a24adabd6a24b7bb5ddd05ef059971f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:12.800355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626457e5-1258-4325-9163-992789350525 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:12.800801 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626457e5-1258-4325-9163-992789350525 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:12.800959 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626457e5-1258-4325-9163-992789350525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:12.821122 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-626457e5-1258-4325-9163-992789350525 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:12.822325 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-626457e5-1258-4325-9163-992789350525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:12.879247 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1297/2588] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:12.888975 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978c014-1557-4f71-9ce9-cd88c0843c22 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:12.889262 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978c014-1557-4f71-9ce9-cd88c0843c22 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:12.889536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978c014-1557-4f71-9ce9-cd88c0843c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:12.909638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-6978c014-1557-4f71-9ce9-cd88c0843c22 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:12.910837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6978c014-1557-4f71-9ce9-cd88c0843c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:12.918726 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1292/2589] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:12.922915 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626457e5-1258-4325-9163-992789350525 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:12.933104 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626457e5-1258-4325-9163-992789350525 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:12.937706 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b2bd3-6b4b-4500-a002-ca5c036d33f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:12.938297 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b2bd3-6b4b-4500-a002-ca5c036d33f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:12.938575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b2bd3-6b4b-4500-a002-ca5c036d33f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:12.938844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b2bd3-6b4b-4500-a002-ca5c036d33f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:12.939479 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b2bd3-6b4b-4500-a002-ca5c036d33f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:31:12.939809 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b2bd3-6b4b-4500-a002-ca5c036d33f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:12.941386 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b2bd3-6b4b-4500-a002-ca5c036d33f4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:12.953119 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1298/2590] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:31:12.957206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978c014-1557-4f71-9ce9-cd88c0843c22 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:12.966261 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978c014-1557-4f71-9ce9-cd88c0843c22 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:12.970952 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b9470-25de-42a5-bdf8-a7b81b575c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:12.971569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b9470-25de-42a5-bdf8-a7b81b575c63 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:12.971853 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b9470-25de-42a5-bdf8-a7b81b575c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:12.972123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b9470-25de-42a5-bdf8-a7b81b575c63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:12.972780 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b9470-25de-42a5-bdf8-a7b81b575c63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:12.974654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b9470-25de-42a5-bdf8-a7b81b575c63 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:13.002222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1293/2591] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:31:12 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:31:13.006354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b2bd3-6b4b-4500-a002-ca5c036d33f4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:13.016133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b2bd3-6b4b-4500-a002-ca5c036d33f4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:13.071154 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab47064-d3be-42d4-a392-eb65e0ffaa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:13.071774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab47064-d3be-42d4-a392-eb65e0ffaa67 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:13.072055 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab47064-d3be-42d4-a392-eb65e0ffaa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:13.072385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab47064-d3be-42d4-a392-eb65e0ffaa67 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:13.073016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab47064-d3be-42d4-a392-eb65e0ffaa67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:13.074916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab47064-d3be-42d4-a392-eb65e0ffaa67 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:13.096008 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1299/2592] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:31:13 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:31:13.100067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b9470-25de-42a5-bdf8-a7b81b575c63 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:13.110943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b9470-25de-42a5-bdf8-a7b81b575c63 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:13.115800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0b5fa-1a96-4972-93d3-0e3f8a294f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:13.116496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0b5fa-1a96-4972-93d3-0e3f8a294f47 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:13.116776 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0b5fa-1a96-4972-93d3-0e3f8a294f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:13.117046 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0b5fa-1a96-4972-93d3-0e3f8a294f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:13.117717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0b5fa-1a96-4972-93d3-0e3f8a294f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:13.119285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0b5fa-1a96-4972-93d3-0e3f8a294f47 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:13.128969 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1294/2593] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:31:13.133277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab47064-d3be-42d4-a392-eb65e0ffaa67 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:13.142806 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab47064-d3be-42d4-a392-eb65e0ffaa67 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:13.148237 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4aee1-1bd6-4a08-ad01-15d81d757418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:13.148831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4aee1-1bd6-4a08-ad01-15d81d757418 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:13.149108 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4aee1-1bd6-4a08-ad01-15d81d757418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:13.149379 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4aee1-1bd6-4a08-ad01-15d81d757418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:13.150006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac4aee1-1bd6-4a08-ad01-15d81d757418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:13.151596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac4aee1-1bd6-4a08-ad01-15d81d757418 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:13.162351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1300/2594] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:31:13.167058 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0b5fa-1a96-4972-93d3-0e3f8a294f47 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:13.176994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0b5fa-1a96-4972-93d3-0e3f8a294f47 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:13.182207 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad62393-c4a9-4af4-9a23-795ea88e35da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:13.182842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad62393-c4a9-4af4-9a23-795ea88e35da admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:13.183123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad62393-c4a9-4af4-9a23-795ea88e35da admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:13.183401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad62393-c4a9-4af4-9a23-795ea88e35da admin admin] PATH_INFO: `/v3/projects/eee5a116bbd54140bf542b8f9b67b79d/users/5a7d228f591940bf82e5a11025d9d959/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:13.197550 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad62393-c4a9-4af4-9a23-795ea88e35da admin admin] RBAC: Authorizing `identity:create_grant(project_id=eee5a116bbd54140bf542b8f9b67b79d, user_id=5a7d228f591940bf82e5a11025d9d959, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:13.200226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad62393-c4a9-4af4-9a23-795ea88e35da admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:13.212788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1295/2595] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:31:13 2022] PUT /identity/v3/projects/eee5a116bbd54140bf542b8f9b67b79d/users/5a7d228f591940bf82e5a11025d9d959/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:13.222825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bba94d-4078-4318-89d8-6c9bac6bf352 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:13.223109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bba94d-4078-4318-89d8-6c9bac6bf352 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:13.223379 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bba94d-4078-4318-89d8-6c9bac6bf352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:13.253204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d5bba94d-4078-4318-89d8-6c9bac6bf352 None None] MFA Rules not processed for user `5a7d228f591940bf82e5a11025d9d959`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:13.254474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d5bba94d-4078-4318-89d8-6c9bac6bf352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:13.297712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:13.318230 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1301/2596] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:13.329456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad62393-c4a9-4af4-9a23-795ea88e35da admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:13.331149 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2ad62393-c4a9-4af4-9a23-795ea88e35da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:13.386590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad62393-c4a9-4af4-9a23-795ea88e35da admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:13.390388 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b781e9-ba4e-4e54-8969-9f52e77fc452 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:13.391045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b781e9-ba4e-4e54-8969-9f52e77fc452 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:13.391325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b781e9-ba4e-4e54-8969-9f52e77fc452 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:13.391653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b781e9-ba4e-4e54-8969-9f52e77fc452 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:13.400417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b781e9-ba4e-4e54-8969-9f52e77fc452 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:13.401839 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b781e9-ba4e-4e54-8969-9f52e77fc452 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:13.410745 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1296/2597] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:31:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:31:16.398559 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bba94d-4078-4318-89d8-6c9bac6bf352 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:16.459836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bba94d-4078-4318-89d8-6c9bac6bf352 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:16.464316 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81505814-e73d-4228-aa6c-3a2941740e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:16.464932 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81505814-e73d-4228-aa6c-3a2941740e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:16.465210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81505814-e73d-4228-aa6c-3a2941740e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:16.465477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81505814-e73d-4228-aa6c-3a2941740e9b admin admin] PATH_INFO: `/v3/users/5a7d228f591940bf82e5a11025d9d959` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:16.466580 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81505814-e73d-4228-aa6c-3a2941740e9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a7d228f591940bf82e5a11025d9d959)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:16.468519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81505814-e73d-4228-aa6c-3a2941740e9b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:16.513630 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-81505814-e73d-4228-aa6c-3a2941740e9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a7d228f591940bf82e5a11025d9d959'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:16.520534 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-81505814-e73d-4228-aa6c-3a2941740e9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a7d228f591940bf82e5a11025d9d959'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:16.521193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-81505814-e73d-4228-aa6c-3a2941740e9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a7d228f591940bf82e5a11025d9d959'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:16.521784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-81505814-e73d-4228-aa6c-3a2941740e9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a7d228f591940bf82e5a11025d9d959'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:16.528717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-81505814-e73d-4228-aa6c-3a2941740e9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a7d228f591940bf82e5a11025d9d959'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:16.545735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1302/2598] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:31:16 2022] DELETE /identity/v3/users/5a7d228f591940bf82e5a11025d9d959 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:16.550252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646c599-2864-4bed-82a2-3a9a465fd899 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:16.550711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646c599-2864-4bed-82a2-3a9a465fd899 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:16.550824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646c599-2864-4bed-82a2-3a9a465fd899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:16.571391 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-9646c599-2864-4bed-82a2-3a9a465fd899 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:16.572618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9646c599-2864-4bed-82a2-3a9a465fd899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:16.576599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:16.631639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1297/2599] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:31:16.641745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81505814-e73d-4228-aa6c-3a2941740e9b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:16.643139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-81505814-e73d-4228-aa6c-3a2941740e9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:16.706583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81505814-e73d-4228-aa6c-3a2941740e9b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:16.710163 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f511d82-868e-4e0f-a1f2-ce3a6746f9b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:16.710764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f511d82-868e-4e0f-a1f2-ce3a6746f9b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:16.711040 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f511d82-868e-4e0f-a1f2-ce3a6746f9b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:16.711306 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f511d82-868e-4e0f-a1f2-ce3a6746f9b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:16.720130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f511d82-868e-4e0f-a1f2-ce3a6746f9b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:16.721564 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f511d82-868e-4e0f-a1f2-ce3a6746f9b7 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:16.731222 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1303/2600] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:31:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:31:16.986911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646c599-2864-4bed-82a2-3a9a465fd899 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:16.988375 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9646c599-2864-4bed-82a2-3a9a465fd899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:17.003292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646c599-2864-4bed-82a2-3a9a465fd899 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:17.008026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea90dd7-e992-4cf4-a23e-1559579fd508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:17.008658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea90dd7-e992-4cf4-a23e-1559579fd508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:17.008938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea90dd7-e992-4cf4-a23e-1559579fd508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:17.009212 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea90dd7-e992-4cf4-a23e-1559579fd508 admin admin] PATH_INFO: `/v3/projects/eee5a116bbd54140bf542b8f9b67b79d` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:17.010360 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea90dd7-e992-4cf4-a23e-1559579fd508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eee5a116bbd54140bf542b8f9b67b79d)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:17.012278 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea90dd7-e992-4cf4-a23e-1559579fd508 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:17.059747 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1ea90dd7-e992-4cf4-a23e-1559579fd508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eee5a116bbd54140bf542b8f9b67b79d', 'request_id': 'req-1ea90dd7-e992-4cf4-a23e-1559579fd508'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:17.068199 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1ea90dd7-e992-4cf4-a23e-1559579fd508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eee5a116bbd54140bf542b8f9b67b79d', 'request_id': 'req-1ea90dd7-e992-4cf4-a23e-1559579fd508'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:17.074560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1ea90dd7-e992-4cf4-a23e-1559579fd508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eee5a116bbd54140bf542b8f9b67b79d', 'request_id': 'req-1ea90dd7-e992-4cf4-a23e-1559579fd508'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:17.085714 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1ea90dd7-e992-4cf4-a23e-1559579fd508 admin admin] The token cache is being invalidate because project eee5a116bbd54140bf542b8f9b67b79d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:31:17.085990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1ea90dd7-e992-4cf4-a23e-1559579fd508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:17.086630 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1ea90dd7-e992-4cf4-a23e-1559579fd508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:31:17.088189 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1298/2601] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:31:16 2022] DELETE /identity/v3/projects/eee5a116bbd54140bf542b8f9b67b79d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:17.093430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3470e-08af-4cbf-84bb-f7c6d10b2318 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:17.093738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3470e-08af-4cbf-84bb-f7c6d10b2318 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:17.094009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3470e-08af-4cbf-84bb-f7c6d10b2318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:17.108078 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:17.114712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-17d3470e-08af-4cbf-84bb-f7c6d10b2318 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:17.116618 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-17d3470e-08af-4cbf-84bb-f7c6d10b2318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:17.174354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1304/2602] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:17.182123 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ffb03-dff2-443a-b2cc-cc59eed508e1 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:17.182407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ffb03-dff2-443a-b2cc-cc59eed508e1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:17.182678 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ffb03-dff2-443a-b2cc-cc59eed508e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:17.202437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-825ffb03-dff2-443a-b2cc-cc59eed508e1 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:17.203671 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-825ffb03-dff2-443a-b2cc-cc59eed508e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:17.211990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1299/2603] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:17.216310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3470e-08af-4cbf-84bb-f7c6d10b2318 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:17.225741 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3470e-08af-4cbf-84bb-f7c6d10b2318 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:17.230774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a4151-5c56-4c74-8388-beb1880044f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:17.231390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a4151-5c56-4c74-8388-beb1880044f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:17.231697 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a4151-5c56-4c74-8388-beb1880044f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:17.231986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a4151-5c56-4c74-8388-beb1880044f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:17.232600 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a4151-5c56-4c74-8388-beb1880044f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:31:17.232898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a4151-5c56-4c74-8388-beb1880044f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:17.234466 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a4151-5c56-4c74-8388-beb1880044f6 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:17.245705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1305/2604] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:31:17.250182 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ffb03-dff2-443a-b2cc-cc59eed508e1 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:17.259364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ffb03-dff2-443a-b2cc-cc59eed508e1 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:17.264056 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d533af2-0d4c-4305-b353-82a0fe206972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:17.264651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d533af2-0d4c-4305-b353-82a0fe206972 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:17.264928 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d533af2-0d4c-4305-b353-82a0fe206972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:17.265197 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d533af2-0d4c-4305-b353-82a0fe206972 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:17.265819 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d533af2-0d4c-4305-b353-82a0fe206972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:17.267714 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d533af2-0d4c-4305-b353-82a0fe206972 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:17.295493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1300/2605] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:31:17 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:31:17.299850 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a4151-5c56-4c74-8388-beb1880044f6 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:17.309648 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a4151-5c56-4c74-8388-beb1880044f6 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:17.364736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231a17c-4cf4-486b-93c7-dbeed19918a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:17.365339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231a17c-4cf4-486b-93c7-dbeed19918a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:17.365620 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231a17c-4cf4-486b-93c7-dbeed19918a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:17.365893 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231a17c-4cf4-486b-93c7-dbeed19918a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:17.367029 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231a17c-4cf4-486b-93c7-dbeed19918a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:17.368961 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231a17c-4cf4-486b-93c7-dbeed19918a1 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:17.389582 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1306/2606] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:31:17 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:31:17.393833 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d533af2-0d4c-4305-b353-82a0fe206972 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:17.404196 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d533af2-0d4c-4305-b353-82a0fe206972 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:17.408819 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2eec6-7a0a-4249-80e7-427b9206ba3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:17.409407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2eec6-7a0a-4249-80e7-427b9206ba3f admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:17.409687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2eec6-7a0a-4249-80e7-427b9206ba3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:17.409958 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2eec6-7a0a-4249-80e7-427b9206ba3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:17.410594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2eec6-7a0a-4249-80e7-427b9206ba3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:17.412197 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2eec6-7a0a-4249-80e7-427b9206ba3f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:17.421827 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1301/2607] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:31:17.425524 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231a17c-4cf4-486b-93c7-dbeed19918a1 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:17.435921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231a17c-4cf4-486b-93c7-dbeed19918a1 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:17.440585 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5add48b-4456-41fd-9244-7f0ce066ad21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:17.441174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5add48b-4456-41fd-9244-7f0ce066ad21 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:17.441457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5add48b-4456-41fd-9244-7f0ce066ad21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:17.441732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5add48b-4456-41fd-9244-7f0ce066ad21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:17.442369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5add48b-4456-41fd-9244-7f0ce066ad21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:17.444019 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5add48b-4456-41fd-9244-7f0ce066ad21 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:17.454962 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1307/2608] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:31:17.458832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2eec6-7a0a-4249-80e7-427b9206ba3f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:17.468507 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2eec6-7a0a-4249-80e7-427b9206ba3f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:17.472997 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2fa22-d464-491e-9ca4-1c28b5f76c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:17.473636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2fa22-d464-491e-9ca4-1c28b5f76c2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:17.473902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2fa22-d464-491e-9ca4-1c28b5f76c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:17.474174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2fa22-d464-491e-9ca4-1c28b5f76c2e admin admin] PATH_INFO: `/v3/projects/1207aabc708d4c1fba0e51581e802ba9/users/04e340005b6b47fd9189ac68c7228196/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:17.488786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2fa22-d464-491e-9ca4-1c28b5f76c2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1207aabc708d4c1fba0e51581e802ba9, user_id=04e340005b6b47fd9189ac68c7228196, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:17.492274 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2fa22-d464-491e-9ca4-1c28b5f76c2e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:17.504598 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1302/2609] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:31:17 2022] PUT /identity/v3/projects/1207aabc708d4c1fba0e51581e802ba9/users/04e340005b6b47fd9189ac68c7228196/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:31:17.515788 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e62ba-f440-4237-980f-2a7e4b69ab59 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:17.516078 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e62ba-f440-4237-980f-2a7e4b69ab59 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:17.516375 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e62ba-f440-4237-980f-2a7e4b69ab59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:17.544316 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-158e62ba-f440-4237-980f-2a7e4b69ab59 None None] MFA Rules not processed for user `04e340005b6b47fd9189ac68c7228196`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:31:17.545535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-158e62ba-f440-4237-980f-2a7e4b69ab59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:17.608642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1308/2610] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:31:17.620637 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2fa22-d464-491e-9ca4-1c28b5f76c2e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:31:17.622378 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c5e2fa22-d464-491e-9ca4-1c28b5f76c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:31:17.653499 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:17.680262 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2fa22-d464-491e-9ca4-1c28b5f76c2e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:31:17.683911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b245c18-1d85-4a55-b96f-08aa817cde9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:31:17.684510 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b245c18-1d85-4a55-b96f-08aa817cde9e service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:31:17.684792 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b245c18-1d85-4a55-b96f-08aa817cde9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:31:17.685065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b245c18-1d85-4a55-b96f-08aa817cde9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:31:17.693309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b245c18-1d85-4a55-b96f-08aa817cde9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:31:17.694725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b245c18-1d85-4a55-b96f-08aa817cde9e service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:31:17.703522 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1303/2611] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:31:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:31:27.605816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:31:27.699461 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=13 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:32:02.183303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e62ba-f440-4237-980f-2a7e4b69ab59 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:02.245743 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e62ba-f440-4237-980f-2a7e4b69ab59 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:02.250686 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f313e52-c6e0-47b2-8a65-c4327c06f180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:02.251458 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f313e52-c6e0-47b2-8a65-c4327c06f180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:02.251767 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f313e52-c6e0-47b2-8a65-c4327c06f180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:02.252043 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f313e52-c6e0-47b2-8a65-c4327c06f180 admin admin] PATH_INFO: `/v3/users/04e340005b6b47fd9189ac68c7228196` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:02.253170 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f313e52-c6e0-47b2-8a65-c4327c06f180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04e340005b6b47fd9189ac68c7228196)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:02.255078 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f313e52-c6e0-47b2-8a65-c4327c06f180 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:02.305287 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1f313e52-c6e0-47b2-8a65-c4327c06f180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04e340005b6b47fd9189ac68c7228196'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:02.312816 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1f313e52-c6e0-47b2-8a65-c4327c06f180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04e340005b6b47fd9189ac68c7228196'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:02.313474 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1f313e52-c6e0-47b2-8a65-c4327c06f180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04e340005b6b47fd9189ac68c7228196'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:02.314080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1f313e52-c6e0-47b2-8a65-c4327c06f180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04e340005b6b47fd9189ac68c7228196'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:02.321006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-1f313e52-c6e0-47b2-8a65-c4327c06f180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04e340005b6b47fd9189ac68c7228196'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:02.338688 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1309/2612] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:32:02 2022] DELETE /identity/v3/users/04e340005b6b47fd9189ac68c7228196 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:32:02.343179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f0ec3-30f4-40d3-9904-27cd014aac4d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:02.343703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f0ec3-30f4-40d3-9904-27cd014aac4d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:02.343822 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f0ec3-30f4-40d3-9904-27cd014aac4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:02.361940 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-de6f0ec3-30f4-40d3-9904-27cd014aac4d None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:32:02.363113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-de6f0ec3-30f4-40d3-9904-27cd014aac4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:02.419339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1304/2613] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:32:02.428796 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f313e52-c6e0-47b2-8a65-c4327c06f180 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:02.430166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1f313e52-c6e0-47b2-8a65-c4327c06f180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:02.486429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f313e52-c6e0-47b2-8a65-c4327c06f180 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:02.490142 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cb9b1-24d6-4cc5-8145-f730fde6ced5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:02.490738 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cb9b1-24d6-4cc5-8145-f730fde6ced5 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:02.491046 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cb9b1-24d6-4cc5-8145-f730fde6ced5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:02.491360 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cb9b1-24d6-4cc5-8145-f730fde6ced5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:02.500779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cb9b1-24d6-4cc5-8145-f730fde6ced5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:02.502190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cb9b1-24d6-4cc5-8145-f730fde6ced5 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:02.511322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1310/2614] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:32:02.771765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f0ec3-30f4-40d3-9904-27cd014aac4d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:02.773592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-de6f0ec3-30f4-40d3-9904-27cd014aac4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:02.786093 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f0ec3-30f4-40d3-9904-27cd014aac4d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:02.795415 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:02.796145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:02.796487 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:02.796824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508 admin admin] PATH_INFO: `/v3/projects/1207aabc708d4c1fba0e51581e802ba9` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:02.798036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1207aabc708d4c1fba0e51581e802ba9)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:02.800037 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:02.845268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1207aabc708d4c1fba0e51581e802ba9', 'request_id': 'req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:02.852484 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1207aabc708d4c1fba0e51581e802ba9', 'request_id': 'req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:02.859390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1207aabc708d4c1fba0e51581e802ba9', 'request_id': 'req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:02.869700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508 admin admin] The token cache is being invalidate because project 1207aabc708d4c1fba0e51581e802ba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:32:02.869990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:02.870627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:02.872152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1305/2615] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:32:02 2022] DELETE /identity/v3/projects/1207aabc708d4c1fba0e51581e802ba9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:32:12.507197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:32:12.867339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:32:24.659091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cb9b1-24d6-4cc5-8145-f730fde6ced5 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:24.660520 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2a1cb9b1-24d6-4cc5-8145-f730fde6ced5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:24.715395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cb9b1-24d6-4cc5-8145-f730fde6ced5 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:24.762513 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f66ef5-a701-493f-b423-ec5b47a11fb5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a889616f86346f48b7573969174ee57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:24.763112 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f66ef5-a701-493f-b423-ec5b47a11fb5 service glance] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:24.763391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f66ef5-a701-493f-b423-ec5b47a11fb5 service glance] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:24.763689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f66ef5-a701-493f-b423-ec5b47a11fb5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:24.765403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-25f66ef5-a701-493f-b423-ec5b47a11fb5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:24.834832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f66ef5-a701-493f-b423-ec5b47a11fb5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:24.836189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f66ef5-a701-493f-b423-ec5b47a11fb5 service glance] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:24.846838 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1311/2616] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:32:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:32:25.016390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:25.017902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:25.074525 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ed6db-3e53-47e1-aac7-66d9a0b56508 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:25.078241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efa759-8dcc-4b3c-89a2-de1e9a595deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:25.078845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efa759-8dcc-4b3c-89a2-de1e9a595deb service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:25.079128 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efa759-8dcc-4b3c-89a2-de1e9a595deb service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:25.079401 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efa759-8dcc-4b3c-89a2-de1e9a595deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:25.081139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-35efa759-8dcc-4b3c-89a2-de1e9a595deb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:25.096015 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35efa759-8dcc-4b3c-89a2-de1e9a595deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:25.097413 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35efa759-8dcc-4b3c-89a2-de1e9a595deb service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:25.108440 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1306/2617] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:32:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:32:34.842132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=74 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:32:35.103597 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=26 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:32:42.402817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f66ef5-a701-493f-b423-ec5b47a11fb5 service glance] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:42.412309 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f66ef5-a701-493f-b423-ec5b47a11fb5 service glance] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:42.415933 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68d0b2-d99b-4e18-b1be-213affaf781d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:42.417185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68d0b2-d99b-4e18-b1be-213affaf781d service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:42.417561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68d0b2-d99b-4e18-b1be-213affaf781d service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:42.417834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68d0b2-d99b-4e18-b1be-213affaf781d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:42.419687 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5b68d0b2-d99b-4e18-b1be-213affaf781d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:42.484185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68d0b2-d99b-4e18-b1be-213affaf781d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:42.485684 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68d0b2-d99b-4e18-b1be-213affaf781d service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:42.496194 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1312/2618] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:32:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:32:45.881534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efa759-8dcc-4b3c-89a2-de1e9a595deb service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:45.883740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-35efa759-8dcc-4b3c-89a2-de1e9a595deb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:45.903181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efa759-8dcc-4b3c-89a2-de1e9a595deb service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:45.907870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e1813-103e-4d4d-9de9-685324f45b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:45.908489 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e1813-103e-4d4d-9de9-685324f45b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:45.908766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e1813-103e-4d4d-9de9-685324f45b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:45.909036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e1813-103e-4d4d-9de9-685324f45b5d admin admin] PATH_INFO: `/v3/users/3066c1c9c79942c289b733d886d9fd0f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:45.923375 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e1813-103e-4d4d-9de9-685324f45b5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3066c1c9c79942c289b733d886d9fd0f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:45.925358 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e1813-103e-4d4d-9de9-685324f45b5d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:45.972036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ec1e1813-103e-4d4d-9de9-685324f45b5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3066c1c9c79942c289b733d886d9fd0f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:45.979077 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ec1e1813-103e-4d4d-9de9-685324f45b5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3066c1c9c79942c289b733d886d9fd0f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:45.979788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ec1e1813-103e-4d4d-9de9-685324f45b5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3066c1c9c79942c289b733d886d9fd0f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:45.980420 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ec1e1813-103e-4d4d-9de9-685324f45b5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3066c1c9c79942c289b733d886d9fd0f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:45.988428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ec1e1813-103e-4d4d-9de9-685324f45b5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3066c1c9c79942c289b733d886d9fd0f'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:46.006776 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1307/2619] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:32:45 2022] DELETE /identity/v3/users/3066c1c9c79942c289b733d886d9fd0f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:32:46.144783 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e1094-b02e-43bd-a5c7-399531e495af None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:46.145338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e1094-b02e-43bd-a5c7-399531e495af None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:46.145338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e1094-b02e-43bd-a5c7-399531e495af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:46.191721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e1813-103e-4d4d-9de9-685324f45b5d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:46.193283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ec1e1813-103e-4d4d-9de9-685324f45b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:46.199421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-233e1094-b02e-43bd-a5c7-399531e495af None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:32:46.201075 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-233e1094-b02e-43bd-a5c7-399531e495af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:46.272725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e1813-103e-4d4d-9de9-685324f45b5d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:46.273952 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1313/2620] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Oct 6 12:32:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:32:46.277722 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c4204-8e8d-4298-ac31-b3c18e4c74c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:46.278350 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c4204-8e8d-4298-ac31-b3c18e4c74c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:46.278646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c4204-8e8d-4298-ac31-b3c18e4c74c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:46.278891 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c4204-8e8d-4298-ac31-b3c18e4c74c7 admin admin] PATH_INFO: `/v3/projects/828d3459cdd641d98772383edfe3268f` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:46.289803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c4204-8e8d-4298-ac31-b3c18e4c74c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=828d3459cdd641d98772383edfe3268f)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:46.291754 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c4204-8e8d-4298-ac31-b3c18e4c74c7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:46.326714 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e1094-b02e-43bd-a5c7-399531e495af None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:46.328156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-233e1094-b02e-43bd-a5c7-399531e495af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:46.342189 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ad5c4204-8e8d-4298-ac31-b3c18e4c74c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '828d3459cdd641d98772383edfe3268f', 'request_id': 'req-ad5c4204-8e8d-4298-ac31-b3c18e4c74c7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:46.345916 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:32:46.351011 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ad5c4204-8e8d-4298-ac31-b3c18e4c74c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '828d3459cdd641d98772383edfe3268f', 'request_id': 'req-ad5c4204-8e8d-4298-ac31-b3c18e4c74c7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:46.358831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ad5c4204-8e8d-4298-ac31-b3c18e4c74c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '828d3459cdd641d98772383edfe3268f', 'request_id': 'req-ad5c4204-8e8d-4298-ac31-b3c18e4c74c7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:46.369252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ad5c4204-8e8d-4298-ac31-b3c18e4c74c7 admin admin] The token cache is being invalidate because project 828d3459cdd641d98772383edfe3268f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:32:46.370056 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ad5c4204-8e8d-4298-ac31-b3c18e4c74c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:46.370692 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ad5c4204-8e8d-4298-ac31-b3c18e4c74c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:32:46.372199 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1308/2621] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:32:46 2022] DELETE /identity/v3/projects/828d3459cdd641d98772383edfe3268f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:32:46.378051 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678ae81-4eef-4c93-9acf-db52c77d8f8f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:46.378337 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678ae81-4eef-4c93-9acf-db52c77d8f8f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:46.378612 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678ae81-4eef-4c93-9acf-db52c77d8f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:46.396759 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:32:46.397898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-b678ae81-4eef-4c93-9acf-db52c77d8f8f None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:32:46.399119 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b678ae81-4eef-4c93-9acf-db52c77d8f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:46.422311 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e1094-b02e-43bd-a5c7-399531e495af None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:46.466377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1309/2622] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:32:46.468050 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f91ae-0e3b-43ff-9cdb-36703091616f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:46.468377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f91ae-0e3b-43ff-9cdb-36703091616f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:46.468643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f91ae-0e3b-43ff-9cdb-36703091616f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:46.488996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-4f5f91ae-0e3b-43ff-9cdb-36703091616f None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:32:46.489199 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248cc47-6d58-4115-a02f-b93982ae7cbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:46.489767 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248cc47-6d58-4115-a02f-b93982ae7cbf service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:46.490106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4f5f91ae-0e3b-43ff-9cdb-36703091616f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:46.490211 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248cc47-6d58-4115-a02f-b93982ae7cbf service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:46.490499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248cc47-6d58-4115-a02f-b93982ae7cbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:46.492330 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9248cc47-6d58-4115-a02f-b93982ae7cbf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:46.498553 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1310/2623] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:32:46.502872 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f91ae-0e3b-43ff-9cdb-36703091616f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:46.511933 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f91ae-0e3b-43ff-9cdb-36703091616f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:46.516805 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d6e09-480b-4e78-9b9b-917c0a209f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:46.517808 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d6e09-480b-4e78-9b9b-917c0a209f7d admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:46.517808 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d6e09-480b-4e78-9b9b-917c0a209f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:46.518042 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d6e09-480b-4e78-9b9b-917c0a209f7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:46.518671 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d6e09-480b-4e78-9b9b-917c0a209f7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:32:46.519146 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d6e09-480b-4e78-9b9b-917c0a209f7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:46.522196 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d6e09-480b-4e78-9b9b-917c0a209f7d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:46.535202 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1311/2624] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:32:46.539417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d6e09-480b-4e78-9b9b-917c0a209f7d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:46.549422 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d6e09-480b-4e78-9b9b-917c0a209f7d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:46.550720 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9248cc47-6d58-4115-a02f-b93982ae7cbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:46.552349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9248cc47-6d58-4115-a02f-b93982ae7cbf service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:46.554027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f84eb-3442-43ec-915c-ffb5f9b9ab3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:46.554625 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f84eb-3442-43ec-915c-ffb5f9b9ab3c admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:46.554907 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f84eb-3442-43ec-915c-ffb5f9b9ab3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:46.555177 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f84eb-3442-43ec-915c-ffb5f9b9ab3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:46.555832 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f84eb-3442-43ec-915c-ffb5f9b9ab3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:46.558299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f84eb-3442-43ec-915c-ffb5f9b9ab3c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:46.560998 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1314/2625] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:32:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 06 12:32:46.612144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1312/2626] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:32:46 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:32:46.616518 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248cc47-6d58-4115-a02f-b93982ae7cbf service swift] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:46.628201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248cc47-6d58-4115-a02f-b93982ae7cbf service swift] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:46.700966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2aedfd-ba70-47cc-b3df-e885f6079239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:46.701577 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2aedfd-ba70-47cc-b3df-e885f6079239 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:46.701856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2aedfd-ba70-47cc-b3df-e885f6079239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:46.702125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2aedfd-ba70-47cc-b3df-e885f6079239 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:46.702767 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2aedfd-ba70-47cc-b3df-e885f6079239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:46.704795 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2aedfd-ba70-47cc-b3df-e885f6079239 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:46.731830 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1315/2627] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:32:46 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:32:46.735589 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f84eb-3442-43ec-915c-ffb5f9b9ab3c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:46.754004 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f84eb-3442-43ec-915c-ffb5f9b9ab3c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:46.765306 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064f0d0-e65f-4515-a8f6-405d81555e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:46.766124 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064f0d0-e65f-4515-a8f6-405d81555e3e admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:46.766512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064f0d0-e65f-4515-a8f6-405d81555e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:46.767044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064f0d0-e65f-4515-a8f6-405d81555e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:46.767886 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064f0d0-e65f-4515-a8f6-405d81555e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:46.769941 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064f0d0-e65f-4515-a8f6-405d81555e3e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:46.781564 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1313/2628] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:32:46.785341 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2aedfd-ba70-47cc-b3df-e885f6079239 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:46.801730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2aedfd-ba70-47cc-b3df-e885f6079239 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:46.810473 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbefdcee-40b0-4661-9887-7fa63e4b07ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:46.811205 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefdcee-40b0-4661-9887-7fa63e4b07ad admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:46.811611 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefdcee-40b0-4661-9887-7fa63e4b07ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:46.811960 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefdcee-40b0-4661-9887-7fa63e4b07ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:46.812780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefdcee-40b0-4661-9887-7fa63e4b07ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:46.814664 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefdcee-40b0-4661-9887-7fa63e4b07ad admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:46.826430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1316/2629] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:32:46.831297 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064f0d0-e65f-4515-a8f6-405d81555e3e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:46.846179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064f0d0-e65f-4515-a8f6-405d81555e3e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:46.851159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c25d3-08d3-4b24-9cdf-02c5d17a7e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:46.851893 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c25d3-08d3-4b24-9cdf-02c5d17a7e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:46.852181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c25d3-08d3-4b24-9cdf-02c5d17a7e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:46.852588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c25d3-08d3-4b24-9cdf-02c5d17a7e19 admin admin] PATH_INFO: `/v3/projects/bee9f30f6e1d4d1ab2c4fe269d78d768/users/404c5e2b85454e1e91b90fa45796511e/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:46.889994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c25d3-08d3-4b24-9cdf-02c5d17a7e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bee9f30f6e1d4d1ab2c4fe269d78d768, user_id=404c5e2b85454e1e91b90fa45796511e, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:46.894033 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c25d3-08d3-4b24-9cdf-02c5d17a7e19 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:46.910170 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1314/2630] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:32:46 2022] PUT /identity/v3/projects/bee9f30f6e1d4d1ab2c4fe269d78d768/users/404c5e2b85454e1e91b90fa45796511e/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:32:46.915352 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00d863-af97-484b-8184-369efb49539c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:46.915745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00d863-af97-484b-8184-369efb49539c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:46.916028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00d863-af97-484b-8184-369efb49539c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:46.939124 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9b00d863-af97-484b-8184-369efb49539c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:32:46.940394 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9b00d863-af97-484b-8184-369efb49539c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:46.944838 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:32:47.011233 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1317/2631] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:32:47.021025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c25d3-08d3-4b24-9cdf-02c5d17a7e19 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:47.023054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-409c25d3-08d3-4b24-9cdf-02c5d17a7e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:47.111347 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c25d3-08d3-4b24-9cdf-02c5d17a7e19 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:47.120177 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae08109-2839-4ced-8fbd-99f5323cc159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:47.121057 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae08109-2839-4ced-8fbd-99f5323cc159 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:47.121437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae08109-2839-4ced-8fbd-99f5323cc159 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:47.121809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae08109-2839-4ced-8fbd-99f5323cc159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:47.137238 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae08109-2839-4ced-8fbd-99f5323cc159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:47.139596 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae08109-2839-4ced-8fbd-99f5323cc159 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:47.147393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:32:47.157344 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1315/2632] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:32:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:32:52.582460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74529-5bd9-4cbc-8851-55e97e504d60 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:52.582993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74529-5bd9-4cbc-8851-55e97e504d60 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:52.582993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74529-5bd9-4cbc-8851-55e97e504d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:52.619241 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d4e74529-5bd9-4cbc-8851-55e97e504d60 None None] MFA Rules not processed for user `404c5e2b85454e1e91b90fa45796511e`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:32:52.620584 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d4e74529-5bd9-4cbc-8851-55e97e504d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:52.727748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1318/2633] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:32:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:32:52.748877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae08109-2839-4ced-8fbd-99f5323cc159 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:52.750581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-bae08109-2839-4ced-8fbd-99f5323cc159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:52.863096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae08109-2839-4ced-8fbd-99f5323cc159 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:52.869603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fca0e-1f3b-4e12-abb8-5346bbdabb8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:52.870655 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fca0e-1f3b-4e12-abb8-5346bbdabb8f service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:52.871154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fca0e-1f3b-4e12-abb8-5346bbdabb8f service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:52.871782 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fca0e-1f3b-4e12-abb8-5346bbdabb8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:52.883734 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fca0e-1f3b-4e12-abb8-5346bbdabb8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:52.885771 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fca0e-1f3b-4e12-abb8-5346bbdabb8f service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:52.909970 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1316/2634] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:32:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:32:54.959446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc83a6-6a09-4e25-b2f6-e287f0c5e7cc None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:54.959952 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc83a6-6a09-4e25-b2f6-e287f0c5e7cc None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:54.960039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc83a6-6a09-4e25-b2f6-e287f0c5e7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:54.987873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-9fdc83a6-6a09-4e25-b2f6-e287f0c5e7cc None None] MFA Rules not processed for user `404c5e2b85454e1e91b90fa45796511e`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:32:54.990073 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9fdc83a6-6a09-4e25-b2f6-e287f0c5e7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:55.007181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1319/2635] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:32:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:32:57.044076 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fca0e-1f3b-4e12-abb8-5346bbdabb8f service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:57.045588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c70fca0e-1f3b-4e12-abb8-5346bbdabb8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:57.196171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fca0e-1f3b-4e12-abb8-5346bbdabb8f service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:57.204042 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21663e74-85de-470f-964a-402af369f6f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:57.204660 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21663e74-85de-470f-964a-402af369f6f7 service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:57.204943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21663e74-85de-470f-964a-402af369f6f7 service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:57.205217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21663e74-85de-470f-964a-402af369f6f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:57.207284 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-21663e74-85de-470f-964a-402af369f6f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:57.221267 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21663e74-85de-470f-964a-402af369f6f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:57.222897 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21663e74-85de-470f-964a-402af369f6f7 service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:57.247551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1317/2636] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:32:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:32:59.722527 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc83a6-6a09-4e25-b2f6-e287f0c5e7cc None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:32:59.724720 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9fdc83a6-6a09-4e25-b2f6-e287f0c5e7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:59.739373 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc83a6-6a09-4e25-b2f6-e287f0c5e7cc None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:32:59.743415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe50c87-152f-4c0f-b9e0-663fbf0133f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:32:59.744145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe50c87-152f-4c0f-b9e0-663fbf0133f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:32:59.744499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe50c87-152f-4c0f-b9e0-663fbf0133f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:32:59.744846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe50c87-152f-4c0f-b9e0-663fbf0133f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:32:59.746962 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-dbe50c87-152f-4c0f-b9e0-663fbf0133f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:32:59.762679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe50c87-152f-4c0f-b9e0-663fbf0133f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:32:59.764310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe50c87-152f-4c0f-b9e0-663fbf0133f4 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:32:59.773624 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1320/2637] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:32:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:33:01.094824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906be237-61d5-45ed-b069-1dfc5bc20acf None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:33:01.095479 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906be237-61d5-45ed-b069-1dfc5bc20acf None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:33:01.095479 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906be237-61d5-45ed-b069-1dfc5bc20acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:33:01.117205 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-906be237-61d5-45ed-b069-1dfc5bc20acf None None] MFA Rules not processed for user `dff0daad12c143fabd16d40a1939b87e`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:33:01.118403 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-906be237-61d5-45ed-b069-1dfc5bc20acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:33:01.174953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1318/2638] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:33:01.909899 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe50c87-152f-4c0f-b9e0-663fbf0133f4 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:33:01.919997 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe50c87-152f-4c0f-b9e0-663fbf0133f4 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:33:01.923899 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849a049-b5e8-456e-ada7-3e718a70cfe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:33:01.924677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849a049-b5e8-456e-ada7-3e718a70cfe4 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:33:01.924956 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849a049-b5e8-456e-ada7-3e718a70cfe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:33:01.925228 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849a049-b5e8-456e-ada7-3e718a70cfe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:33:01.926960 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e849a049-b5e8-456e-ada7-3e718a70cfe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:33:01.936783 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849a049-b5e8-456e-ada7-3e718a70cfe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:33:01.938241 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849a049-b5e8-456e-ada7-3e718a70cfe4 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:33:01.946611 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1321/2639] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:33:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:33:11.167967 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=91 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:33:11.943781 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=70 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:33:13.556912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf730f-cc16-49df-826e-dcaad27cb8e1 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:33:13.557988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf730f-cc16-49df-826e-dcaad27cb8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:33:13.558077 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf730f-cc16-49df-826e-dcaad27cb8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:33:13.579685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-fbcf730f-cc16-49df-826e-dcaad27cb8e1 None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:33:13.581047 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fbcf730f-cc16-49df-826e-dcaad27cb8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:33:13.641170 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1319/2640] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Oct 6 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:33:13.656458 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849a049-b5e8-456e-ada7-3e718a70cfe4 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:33:13.658565 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e849a049-b5e8-456e-ada7-3e718a70cfe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:33:13.734870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849a049-b5e8-456e-ada7-3e718a70cfe4 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:33:13.739614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d899c-48c4-4535-b173-b8aa48e44b75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:33:13.740341 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d899c-48c4-4535-b173-b8aa48e44b75 service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:33:13.740721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d899c-48c4-4535-b173-b8aa48e44b75 service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:33:13.741088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d899c-48c4-4535-b173-b8aa48e44b75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:33:13.750404 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d899c-48c4-4535-b173-b8aa48e44b75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:33:13.753067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d899c-48c4-4535-b173-b8aa48e44b75 service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:33:13.762829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1322/2641] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:33:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 12:33:23.631782 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=25 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:33:23.758293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=32 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:33:33.791383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf730f-cc16-49df-826e-dcaad27cb8e1 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:33:33.805636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf730f-cc16-49df-826e-dcaad27cb8e1 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:33:33.812701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865aa72-495e-46f8-8cad-7015488209a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:33:33.813412 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865aa72-495e-46f8-8cad-7015488209a7 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:33:33.813854 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865aa72-495e-46f8-8cad-7015488209a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:33:33.813993 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865aa72-495e-46f8-8cad-7015488209a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:33:33.816176 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c865aa72-495e-46f8-8cad-7015488209a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:33:33.830082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865aa72-495e-46f8-8cad-7015488209a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:33:33.831702 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865aa72-495e-46f8-8cad-7015488209a7 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:33:33.844762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1320/2642] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:33:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:33:36.933895 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d899c-48c4-4535-b173-b8aa48e44b75 service swift] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:33:36.943080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d899c-48c4-4535-b173-b8aa48e44b75 service swift] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:33:36.946657 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819261ce-3eb5-4e17-bb98-0e038073f27e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:33:36.947237 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819261ce-3eb5-4e17-bb98-0e038073f27e service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:33:36.947547 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819261ce-3eb5-4e17-bb98-0e038073f27e service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:33:36.947808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819261ce-3eb5-4e17-bb98-0e038073f27e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:33:36.950339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-819261ce-3eb5-4e17-bb98-0e038073f27e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:33:37.012700 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819261ce-3eb5-4e17-bb98-0e038073f27e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:33:37.014126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819261ce-3eb5-4e17-bb98-0e038073f27e service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:33:37.023569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1323/2643] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:33:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:33:38.747953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fb851-bd80-4d94-aa7c-db9dc830d578 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:33:38.748485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fb851-bd80-4d94-aa7c-db9dc830d578 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:33:38.748639 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fb851-bd80-4d94-aa7c-db9dc830d578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:33:38.768276 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-043fb851-bd80-4d94-aa7c-db9dc830d578 None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:33:38.769488 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-043fb851-bd80-4d94-aa7c-db9dc830d578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:33:38.777847 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1321/2644] 10.222.0.33 () {58 vars in 967 bytes} [Thu Oct 6 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:33:38.790568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819261ce-3eb5-4e17-bb98-0e038073f27e service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:33:38.803011 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819261ce-3eb5-4e17-bb98-0e038073f27e service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:33:38.809009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f2cd5-f890-48cd-9680-5368f8001d6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:33:38.809624 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f2cd5-f890-48cd-9680-5368f8001d6d service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:33:38.810306 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f2cd5-f890-48cd-9680-5368f8001d6d service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:33:38.810306 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f2cd5-f890-48cd-9680-5368f8001d6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:33:38.819480 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f2cd5-f890-48cd-9680-5368f8001d6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:33:38.821108 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f2cd5-f890-48cd-9680-5368f8001d6d service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:33:38.830402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1324/2645] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:33:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 12:33:48.763139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=9 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:33:48.826138 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=36 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:34:40.411622 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85686ae8-2dc4-40d3-9d94-1822c46ac7d0 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:40.412140 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85686ae8-2dc4-40d3-9d94-1822c46ac7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:40.412253 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85686ae8-2dc4-40d3-9d94-1822c46ac7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:40.430666 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-85686ae8-2dc4-40d3-9d94-1822c46ac7d0 None None] MFA Rules not processed for user `404c5e2b85454e1e91b90fa45796511e`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:34:40.431890 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-85686ae8-2dc4-40d3-9d94-1822c46ac7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:34:40.439113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1322/2646] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:34:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:34:50.425486 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:34:53.854113 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f2cd5-f890-48cd-9680-5368f8001d6d service swift] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:53.863770 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f2cd5-f890-48cd-9680-5368f8001d6d service swift] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:53.867377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785eae48-98b8-4f21-90df-179d6c83a54d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:53.868074 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785eae48-98b8-4f21-90df-179d6c83a54d service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:53.868410 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785eae48-98b8-4f21-90df-179d6c83a54d service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:53.868699 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785eae48-98b8-4f21-90df-179d6c83a54d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:53.870522 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-785eae48-98b8-4f21-90df-179d6c83a54d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:34:53.883327 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785eae48-98b8-4f21-90df-179d6c83a54d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:53.884786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785eae48-98b8-4f21-90df-179d6c83a54d service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:53.895337 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1325/2647] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:34:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:34:57.046509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85686ae8-2dc4-40d3-9d94-1822c46ac7d0 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:57.048573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-85686ae8-2dc4-40d3-9d94-1822c46ac7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:34:57.062682 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85686ae8-2dc4-40d3-9d94-1822c46ac7d0 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:57.067219 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abc5fd-fcb6-4f4e-b6ce-6ff9e7618098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:57.067867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abc5fd-fcb6-4f4e-b6ce-6ff9e7618098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:57.068144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abc5fd-fcb6-4f4e-b6ce-6ff9e7618098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:57.068414 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abc5fd-fcb6-4f4e-b6ce-6ff9e7618098 admin admin] PATH_INFO: `/v3/users/ff9c115c7083441c8b363e9b685ef110` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:57.081496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abc5fd-fcb6-4f4e-b6ce-6ff9e7618098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff9c115c7083441c8b363e9b685ef110)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:57.083409 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abc5fd-fcb6-4f4e-b6ce-6ff9e7618098 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:57.129982 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-37abc5fd-fcb6-4f4e-b6ce-6ff9e7618098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff9c115c7083441c8b363e9b685ef110'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:57.136760 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-37abc5fd-fcb6-4f4e-b6ce-6ff9e7618098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff9c115c7083441c8b363e9b685ef110'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:57.137426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-37abc5fd-fcb6-4f4e-b6ce-6ff9e7618098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff9c115c7083441c8b363e9b685ef110'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:57.138040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-37abc5fd-fcb6-4f4e-b6ce-6ff9e7618098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff9c115c7083441c8b363e9b685ef110'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:57.145177 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-37abc5fd-fcb6-4f4e-b6ce-6ff9e7618098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff9c115c7083441c8b363e9b685ef110'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:57.163319 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1323/2648] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:34:57 2022] DELETE /identity/v3/users/ff9c115c7083441c8b363e9b685ef110 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:34:57.343459 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785eae48-98b8-4f21-90df-179d6c83a54d service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:57.353804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785eae48-98b8-4f21-90df-179d6c83a54d service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:57.411530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5612e2-3e52-446d-8548-48db97113492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:57.412174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5612e2-3e52-446d-8548-48db97113492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:57.412456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5612e2-3e52-446d-8548-48db97113492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:57.412728 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5612e2-3e52-446d-8548-48db97113492 admin admin] PATH_INFO: `/v3/projects/eda6037b1d4b4d31b7a7ad298452cfbb` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:57.422860 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5612e2-3e52-446d-8548-48db97113492 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eda6037b1d4b4d31b7a7ad298452cfbb)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:57.424788 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5612e2-3e52-446d-8548-48db97113492 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:57.472223 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9b5612e2-3e52-446d-8548-48db97113492 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eda6037b1d4b4d31b7a7ad298452cfbb', 'request_id': 'req-9b5612e2-3e52-446d-8548-48db97113492'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:57.479132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9b5612e2-3e52-446d-8548-48db97113492 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eda6037b1d4b4d31b7a7ad298452cfbb', 'request_id': 'req-9b5612e2-3e52-446d-8548-48db97113492'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:57.485650 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9b5612e2-3e52-446d-8548-48db97113492 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eda6037b1d4b4d31b7a7ad298452cfbb', 'request_id': 'req-9b5612e2-3e52-446d-8548-48db97113492'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:57.496201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9b5612e2-3e52-446d-8548-48db97113492 admin admin] The token cache is being invalidate because project eda6037b1d4b4d31b7a7ad298452cfbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:34:57.496436 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9b5612e2-3e52-446d-8548-48db97113492 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:57.497171 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9b5612e2-3e52-446d-8548-48db97113492 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:57.498659 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1326/2649] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:34:57 2022] DELETE /identity/v3/projects/eda6037b1d4b4d31b7a7ad298452cfbb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:34:57.503581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8b5ec-4185-4097-9f9f-84a2daec0cf9 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:57.503830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8b5ec-4185-4097-9f9f-84a2daec0cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:57.504101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8b5ec-4185-4097-9f9f-84a2daec0cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:57.523634 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-37a8b5ec-4185-4097-9f9f-84a2daec0cf9 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:34:57.524818 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-37a8b5ec-4185-4097-9f9f-84a2daec0cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:34:57.583263 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1324/2650] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:34:57.591043 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edeecf-0d9e-4626-a4db-f9397bc37c7a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:57.591328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edeecf-0d9e-4626-a4db-f9397bc37c7a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:57.591641 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edeecf-0d9e-4626-a4db-f9397bc37c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:57.611560 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-55edeecf-0d9e-4626-a4db-f9397bc37c7a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:34:57.612741 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-55edeecf-0d9e-4626-a4db-f9397bc37c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:34:57.620862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1327/2651] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:34:57.625340 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a8b5ec-4185-4097-9f9f-84a2daec0cf9 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:57.634881 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a8b5ec-4185-4097-9f9f-84a2daec0cf9 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:57.639973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecdcfb8-4b71-4aca-8521-6657e07873fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:57.640562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecdcfb8-4b71-4aca-8521-6657e07873fa admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:57.640839 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecdcfb8-4b71-4aca-8521-6657e07873fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:57.641108 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecdcfb8-4b71-4aca-8521-6657e07873fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:57.641740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecdcfb8-4b71-4aca-8521-6657e07873fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:34:57.642042 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecdcfb8-4b71-4aca-8521-6657e07873fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:57.643644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecdcfb8-4b71-4aca-8521-6657e07873fa admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:57.654784 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1325/2652] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:34:57.659126 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edeecf-0d9e-4626-a4db-f9397bc37c7a None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:57.669032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edeecf-0d9e-4626-a4db-f9397bc37c7a None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:57.673698 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc4fc3-f262-44e2-ab38-ad2ea4d90994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:57.674286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc4fc3-f262-44e2-ab38-ad2ea4d90994 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:57.674564 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc4fc3-f262-44e2-ab38-ad2ea4d90994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:57.674833 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc4fc3-f262-44e2-ab38-ad2ea4d90994 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:57.675456 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc4fc3-f262-44e2-ab38-ad2ea4d90994 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:57.677361 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc4fc3-f262-44e2-ab38-ad2ea4d90994 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:57.704797 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1328/2653] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:34:57 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:34:57.708990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecdcfb8-4b71-4aca-8521-6657e07873fa admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:57.718765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecdcfb8-4b71-4aca-8521-6657e07873fa admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:57.761846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:34:57.775716 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018d0f3-96f2-46ab-b724-658089771400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:57.776313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018d0f3-96f2-46ab-b724-658089771400 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:57.776590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018d0f3-96f2-46ab-b724-658089771400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:57.776858 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018d0f3-96f2-46ab-b724-658089771400 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:57.777517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018d0f3-96f2-46ab-b724-658089771400 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:57.779404 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018d0f3-96f2-46ab-b724-658089771400 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:57.803462 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1326/2654] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:34:57 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:34:57.804649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc4fc3-f262-44e2-ab38-ad2ea4d90994 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:57.815240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc4fc3-f262-44e2-ab38-ad2ea4d90994 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:57.819916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d843c2c-9e42-4fd2-a14a-32f61664bc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:57.820506 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d843c2c-9e42-4fd2-a14a-32f61664bc3a admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:57.820779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d843c2c-9e42-4fd2-a14a-32f61664bc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:57.821049 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d843c2c-9e42-4fd2-a14a-32f61664bc3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:57.821694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d843c2c-9e42-4fd2-a14a-32f61664bc3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:57.823268 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d843c2c-9e42-4fd2-a14a-32f61664bc3a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:57.833548 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1329/2655] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:34:57.837371 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018d0f3-96f2-46ab-b724-658089771400 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:57.847223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018d0f3-96f2-46ab-b724-658089771400 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:57.852015 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e271e08-5416-40cc-8696-9f97045c0ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:57.852606 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e271e08-5416-40cc-8696-9f97045c0ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:57.852885 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e271e08-5416-40cc-8696-9f97045c0ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:57.853154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e271e08-5416-40cc-8696-9f97045c0ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:57.853793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e271e08-5416-40cc-8696-9f97045c0ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:57.855426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e271e08-5416-40cc-8696-9f97045c0ae9 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:57.865766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1327/2656] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:34:57.869921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d843c2c-9e42-4fd2-a14a-32f61664bc3a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:57.880260 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d843c2c-9e42-4fd2-a14a-32f61664bc3a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:57.884848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef7e27-83c1-494b-bd3b-1705e4c360e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:57.885477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef7e27-83c1-494b-bd3b-1705e4c360e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:57.885756 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef7e27-83c1-494b-bd3b-1705e4c360e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:57.886026 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef7e27-83c1-494b-bd3b-1705e4c360e7 admin admin] PATH_INFO: `/v3/projects/1f682fd5c70d4840946b5831fe745f28/users/b695789df9b84f58a3c6714ce2bba35e/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:57.900680 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef7e27-83c1-494b-bd3b-1705e4c360e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f682fd5c70d4840946b5831fe745f28, user_id=b695789df9b84f58a3c6714ce2bba35e, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:57.903290 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef7e27-83c1-494b-bd3b-1705e4c360e7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:57.916457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1330/2657] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:34:57 2022] PUT /identity/v3/projects/1f682fd5c70d4840946b5831fe745f28/users/b695789df9b84f58a3c6714ce2bba35e/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:34:57.927661 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65442a9-7538-4023-94a0-bfa492beee84 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:57.927944 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65442a9-7538-4023-94a0-bfa492beee84 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:57.928212 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65442a9-7538-4023-94a0-bfa492beee84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:57.955755 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e65442a9-7538-4023-94a0-bfa492beee84 None None] MFA Rules not processed for user `b695789df9b84f58a3c6714ce2bba35e`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:34:57.956962 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e65442a9-7538-4023-94a0-bfa492beee84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:34:58.020891 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1328/2658] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:34:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:34:58.025429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef7e27-83c1-494b-bd3b-1705e4c360e7 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:58.031597 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:34:58.089070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef7e27-83c1-494b-bd3b-1705e4c360e7 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:58.093638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc9fb8-1368-44ec-8d48-95b6c95cb9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:58.094235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc9fb8-1368-44ec-8d48-95b6c95cb9e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:58.094533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc9fb8-1368-44ec-8d48-95b6c95cb9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:58.094802 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc9fb8-1368-44ec-8d48-95b6c95cb9e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:58.095420 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc9fb8-1368-44ec-8d48-95b6c95cb9e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:58.097338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc9fb8-1368-44ec-8d48-95b6c95cb9e6 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:58.125119 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1331/2659] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:34:58 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:34:58.129424 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65442a9-7538-4023-94a0-bfa492beee84 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:58.140218 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65442a9-7538-4023-94a0-bfa492beee84 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:58.194953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba7a5f-9f32-4cdc-855e-bdcf2ed73049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:58.195565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba7a5f-9f32-4cdc-855e-bdcf2ed73049 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:58.195846 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba7a5f-9f32-4cdc-855e-bdcf2ed73049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:58.196116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba7a5f-9f32-4cdc-855e-bdcf2ed73049 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:58.196739 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba7a5f-9f32-4cdc-855e-bdcf2ed73049 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:58.198624 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba7a5f-9f32-4cdc-855e-bdcf2ed73049 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:58.219138 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1329/2660] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:34:58 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:34:58.223235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc9fb8-1368-44ec-8d48-95b6c95cb9e6 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:58.233827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc9fb8-1368-44ec-8d48-95b6c95cb9e6 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:58.238344 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebbc7b-cf5b-46a3-813b-d683e2bf8bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:58.238932 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebbc7b-cf5b-46a3-813b-d683e2bf8bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:58.239210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebbc7b-cf5b-46a3-813b-d683e2bf8bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:58.239481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebbc7b-cf5b-46a3-813b-d683e2bf8bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:58.240210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebbc7b-cf5b-46a3-813b-d683e2bf8bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:58.241798 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebbc7b-cf5b-46a3-813b-d683e2bf8bf7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:58.252758 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1332/2661] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:34:58.256898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba7a5f-9f32-4cdc-855e-bdcf2ed73049 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:58.266828 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba7a5f-9f32-4cdc-855e-bdcf2ed73049 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:58.271442 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b25b2d-4970-464e-9f40-c207e8e4fb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:58.272064 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b25b2d-4970-464e-9f40-c207e8e4fb73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:58.272344 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b25b2d-4970-464e-9f40-c207e8e4fb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:58.272617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b25b2d-4970-464e-9f40-c207e8e4fb73 admin admin] PATH_INFO: `/v3/projects/036910105794443abb990b0e12322afa/users/d7ae3754cb52406eb86fab7bbf3c7e39/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:58.287439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b25b2d-4970-464e-9f40-c207e8e4fb73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=036910105794443abb990b0e12322afa, user_id=d7ae3754cb52406eb86fab7bbf3c7e39, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:58.290131 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b25b2d-4970-464e-9f40-c207e8e4fb73 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:58.303524 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1330/2662] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:34:58 2022] PUT /identity/v3/projects/036910105794443abb990b0e12322afa/users/d7ae3754cb52406eb86fab7bbf3c7e39/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:34:58.307596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebbc7b-cf5b-46a3-813b-d683e2bf8bf7 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:58.318101 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebbc7b-cf5b-46a3-813b-d683e2bf8bf7 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:58.372519 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75711754-6668-4c2c-bf6d-c8ff851ed505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:58.373149 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75711754-6668-4c2c-bf6d-c8ff851ed505 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:58.373390 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75711754-6668-4c2c-bf6d-c8ff851ed505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:58.373666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75711754-6668-4c2c-bf6d-c8ff851ed505 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:58.374300 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75711754-6668-4c2c-bf6d-c8ff851ed505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:58.375928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75711754-6668-4c2c-bf6d-c8ff851ed505 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:58.385426 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1333/2663] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:34:58.389544 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b25b2d-4970-464e-9f40-c207e8e4fb73 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:58.399991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b25b2d-4970-464e-9f40-c207e8e4fb73 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:58.405124 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59b93d-0747-482b-acd4-e0ed9a19530a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:58.406041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59b93d-0747-482b-acd4-e0ed9a19530a admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:58.406300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59b93d-0747-482b-acd4-e0ed9a19530a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:58.406571 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59b93d-0747-482b-acd4-e0ed9a19530a admin admin] PATH_INFO: `/v3/projects/036910105794443abb990b0e12322afa/users/d7ae3754cb52406eb86fab7bbf3c7e39/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:58.408560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59b93d-0747-482b-acd4-e0ed9a19530a admin admin] RBAC: Authorizing `identity:create_grant(project_id=036910105794443abb990b0e12322afa, user_id=d7ae3754cb52406eb86fab7bbf3c7e39, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:58.411153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59b93d-0747-482b-acd4-e0ed9a19530a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:58.422415 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1331/2664] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:34:58 2022] PUT /identity/v3/projects/036910105794443abb990b0e12322afa/users/d7ae3754cb52406eb86fab7bbf3c7e39/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:34:58.433919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ae6b9-bffd-4924-bd07-21682ac80157 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:58.434207 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ae6b9-bffd-4924-bd07-21682ac80157 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:58.434484 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ae6b9-bffd-4924-bd07-21682ac80157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:58.461229 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8d4ae6b9-bffd-4924-bd07-21682ac80157 None None] MFA Rules not processed for user `d7ae3754cb52406eb86fab7bbf3c7e39`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:34:58.462460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8d4ae6b9-bffd-4924-bd07-21682ac80157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:34:58.526478 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:34:58.530990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1334/2665] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:34:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:34:58.542042 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59b93d-0747-482b-acd4-e0ed9a19530a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:58.543767 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1f59b93d-0747-482b-acd4-e0ed9a19530a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:34:58.554282 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:34:58.601040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59b93d-0747-482b-acd4-e0ed9a19530a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:58.604677 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d0387-e8cd-4207-8dd3-6fc20a9e1049 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:58.605269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d0387-e8cd-4207-8dd3-6fc20a9e1049 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:58.605543 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d0387-e8cd-4207-8dd3-6fc20a9e1049 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:58.605920 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d0387-e8cd-4207-8dd3-6fc20a9e1049 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:58.614552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d0387-e8cd-4207-8dd3-6fc20a9e1049 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:58.615993 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d0387-e8cd-4207-8dd3-6fc20a9e1049 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:58.625503 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1332/2666] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:34:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:34:59.003067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ae6b9-bffd-4924-bd07-21682ac80157 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:59.063757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ae6b9-bffd-4924-bd07-21682ac80157 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:59.068207 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec2543-c257-40cf-ac23-d37edd39bf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:59.068828 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec2543-c257-40cf-ac23-d37edd39bf91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:59.069110 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec2543-c257-40cf-ac23-d37edd39bf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:59.069427 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec2543-c257-40cf-ac23-d37edd39bf91 admin admin] PATH_INFO: `/v3/users/b695789df9b84f58a3c6714ce2bba35e` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:59.070521 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec2543-c257-40cf-ac23-d37edd39bf91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b695789df9b84f58a3c6714ce2bba35e)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:59.072471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec2543-c257-40cf-ac23-d37edd39bf91 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:59.119266 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0bec2543-c257-40cf-ac23-d37edd39bf91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b695789df9b84f58a3c6714ce2bba35e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:59.126006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0bec2543-c257-40cf-ac23-d37edd39bf91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b695789df9b84f58a3c6714ce2bba35e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:59.126660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0bec2543-c257-40cf-ac23-d37edd39bf91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b695789df9b84f58a3c6714ce2bba35e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:59.127271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0bec2543-c257-40cf-ac23-d37edd39bf91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b695789df9b84f58a3c6714ce2bba35e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:59.134243 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-0bec2543-c257-40cf-ac23-d37edd39bf91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b695789df9b84f58a3c6714ce2bba35e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:59.151033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1335/2667] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:34:59 2022] DELETE /identity/v3/users/b695789df9b84f58a3c6714ce2bba35e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:34:59.156313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e3104-4cc2-4752-9617-8d53a364f493 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:59.156796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e3104-4cc2-4752-9617-8d53a364f493 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:59.156912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e3104-4cc2-4752-9617-8d53a364f493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:59.176028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-dc2e3104-4cc2-4752-9617-8d53a364f493 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:34:59.177235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-dc2e3104-4cc2-4752-9617-8d53a364f493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:34:59.234696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1333/2668] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:34:59.244908 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec2543-c257-40cf-ac23-d37edd39bf91 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:59.245992 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-0bec2543-c257-40cf-ac23-d37edd39bf91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:34:59.302299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec2543-c257-40cf-ac23-d37edd39bf91 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:59.305957 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475037f-3984-4a51-afd2-1343084cf496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:59.306567 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475037f-3984-4a51-afd2-1343084cf496 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:59.306846 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475037f-3984-4a51-afd2-1343084cf496 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:59.307119 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475037f-3984-4a51-afd2-1343084cf496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:59.316169 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475037f-3984-4a51-afd2-1343084cf496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:59.317537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475037f-3984-4a51-afd2-1343084cf496 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:59.327010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1336/2669] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:34:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:34:59.584943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e3104-4cc2-4752-9617-8d53a364f493 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:59.586368 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-dc2e3104-4cc2-4752-9617-8d53a364f493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:34:59.599603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e3104-4cc2-4752-9617-8d53a364f493 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:59.604204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c26b7-69d4-43be-a008-7722a71b0e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:59.604836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c26b7-69d4-43be-a008-7722a71b0e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:59.605113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c26b7-69d4-43be-a008-7722a71b0e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:59.605383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c26b7-69d4-43be-a008-7722a71b0e53 admin admin] PATH_INFO: `/v3/projects/1f682fd5c70d4840946b5831fe745f28` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:59.606508 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c26b7-69d4-43be-a008-7722a71b0e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f682fd5c70d4840946b5831fe745f28)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:59.608445 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c26b7-69d4-43be-a008-7722a71b0e53 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:59.655539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b62c26b7-69d4-43be-a008-7722a71b0e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f682fd5c70d4840946b5831fe745f28', 'request_id': 'req-b62c26b7-69d4-43be-a008-7722a71b0e53'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:59.662798 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b62c26b7-69d4-43be-a008-7722a71b0e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f682fd5c70d4840946b5831fe745f28', 'request_id': 'req-b62c26b7-69d4-43be-a008-7722a71b0e53'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:59.669213 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b62c26b7-69d4-43be-a008-7722a71b0e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f682fd5c70d4840946b5831fe745f28', 'request_id': 'req-b62c26b7-69d4-43be-a008-7722a71b0e53'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:59.679381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b62c26b7-69d4-43be-a008-7722a71b0e53 admin admin] The token cache is being invalidate because project 1f682fd5c70d4840946b5831fe745f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:34:59.679717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b62c26b7-69d4-43be-a008-7722a71b0e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:59.680342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-b62c26b7-69d4-43be-a008-7722a71b0e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:59.681797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1334/2670] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:34:59 2022] DELETE /identity/v3/projects/1f682fd5c70d4840946b5831fe745f28 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:34:59.685860 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475037f-3984-4a51-afd2-1343084cf496 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:34:59.695212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475037f-3984-4a51-afd2-1343084cf496 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:34:59.714005 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:34:59.752407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992aad7-6ff9-435b-b07c-c1df50651b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:34:59.753020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992aad7-6ff9-435b-b07c-c1df50651b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:34:59.753296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992aad7-6ff9-435b-b07c-c1df50651b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:34:59.753561 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992aad7-6ff9-435b-b07c-c1df50651b90 admin admin] PATH_INFO: `/v3/users/d7ae3754cb52406eb86fab7bbf3c7e39` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:34:59.754709 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992aad7-6ff9-435b-b07c-c1df50651b90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7ae3754cb52406eb86fab7bbf3c7e39)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:34:59.756628 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992aad7-6ff9-435b-b07c-c1df50651b90 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:34:59.801973 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a992aad7-6ff9-435b-b07c-c1df50651b90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7ae3754cb52406eb86fab7bbf3c7e39'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:59.809288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a992aad7-6ff9-435b-b07c-c1df50651b90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7ae3754cb52406eb86fab7bbf3c7e39'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:59.809976 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a992aad7-6ff9-435b-b07c-c1df50651b90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7ae3754cb52406eb86fab7bbf3c7e39'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:59.810650 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a992aad7-6ff9-435b-b07c-c1df50651b90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7ae3754cb52406eb86fab7bbf3c7e39'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:59.818788 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-a992aad7-6ff9-435b-b07c-c1df50651b90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7ae3754cb52406eb86fab7bbf3c7e39'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:34:59.836549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1337/2671] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:34:59 2022] DELETE /identity/v3/users/d7ae3754cb52406eb86fab7bbf3c7e39 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:00.104688 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c26b7-69d4-43be-a008-7722a71b0e53 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:00.106318 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-b62c26b7-69d4-43be-a008-7722a71b0e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:00.120025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c26b7-69d4-43be-a008-7722a71b0e53 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:00.153466 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:00.179675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a021e404-269f-4973-90d8-431a9e2d6a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:00.180459 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021e404-269f-4973-90d8-431a9e2d6a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:00.180742 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021e404-269f-4973-90d8-431a9e2d6a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:00.181011 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021e404-269f-4973-90d8-431a9e2d6a47 admin admin] PATH_INFO: `/v3/projects/036910105794443abb990b0e12322afa` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:00.182109 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021e404-269f-4973-90d8-431a9e2d6a47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=036910105794443abb990b0e12322afa)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:00.184647 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021e404-269f-4973-90d8-431a9e2d6a47 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:00.229463 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a021e404-269f-4973-90d8-431a9e2d6a47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '036910105794443abb990b0e12322afa', 'request_id': 'req-a021e404-269f-4973-90d8-431a9e2d6a47'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:00.237340 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a021e404-269f-4973-90d8-431a9e2d6a47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '036910105794443abb990b0e12322afa', 'request_id': 'req-a021e404-269f-4973-90d8-431a9e2d6a47'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:00.243777 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a021e404-269f-4973-90d8-431a9e2d6a47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '036910105794443abb990b0e12322afa', 'request_id': 'req-a021e404-269f-4973-90d8-431a9e2d6a47'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:00.254747 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a021e404-269f-4973-90d8-431a9e2d6a47 admin admin] The token cache is being invalidate because project 036910105794443abb990b0e12322afa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:35:00.255040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a021e404-269f-4973-90d8-431a9e2d6a47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:00.255730 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a021e404-269f-4973-90d8-431a9e2d6a47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:00.257244 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1335/2672] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:35:00 2022] DELETE /identity/v3/projects/036910105794443abb990b0e12322afa => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:00.262109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eed4ef-af0c-4c4f-b5e1-2cbcbb2d2d4a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:00.262415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eed4ef-af0c-4c4f-b5e1-2cbcbb2d2d4a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:00.262687 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eed4ef-af0c-4c4f-b5e1-2cbcbb2d2d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:00.282047 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-77eed4ef-af0c-4c4f-b5e1-2cbcbb2d2d4a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:00.283220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-77eed4ef-af0c-4c4f-b5e1-2cbcbb2d2d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:00.341048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1338/2673] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:35:00.348838 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fd9e2-96ac-4fb1-98d1-92e335eca222 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:00.349065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fd9e2-96ac-4fb1-98d1-92e335eca222 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:00.349341 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fd9e2-96ac-4fb1-98d1-92e335eca222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:00.369217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-431fd9e2-96ac-4fb1-98d1-92e335eca222 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:00.370411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-431fd9e2-96ac-4fb1-98d1-92e335eca222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:00.378843 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1336/2674] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:35:00.382815 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eed4ef-af0c-4c4f-b5e1-2cbcbb2d2d4a None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:00.392381 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eed4ef-af0c-4c4f-b5e1-2cbcbb2d2d4a None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:00.397000 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22d8d6-6a93-4953-bee4-8e199c1b9b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:00.397576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22d8d6-6a93-4953-bee4-8e199c1b9b9e admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:00.397854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22d8d6-6a93-4953-bee4-8e199c1b9b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:00.398125 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22d8d6-6a93-4953-bee4-8e199c1b9b9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:00.398758 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22d8d6-6a93-4953-bee4-8e199c1b9b9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:35:00.399054 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22d8d6-6a93-4953-bee4-8e199c1b9b9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:00.400654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22d8d6-6a93-4953-bee4-8e199c1b9b9e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:00.412000 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1339/2675] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:35:00.416248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fd9e2-96ac-4fb1-98d1-92e335eca222 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:00.425821 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fd9e2-96ac-4fb1-98d1-92e335eca222 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:00.430357 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149f090-631e-4152-b143-2633378a8628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:00.430944 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149f090-631e-4152-b143-2633378a8628 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:00.431225 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149f090-631e-4152-b143-2633378a8628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:00.431494 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149f090-631e-4152-b143-2633378a8628 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:00.432140 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149f090-631e-4152-b143-2633378a8628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:00.434568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149f090-631e-4152-b143-2633378a8628 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:00.461432 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1337/2676] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:35:00 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:35:00.465730 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22d8d6-6a93-4953-bee4-8e199c1b9b9e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:00.475826 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22d8d6-6a93-4953-bee4-8e199c1b9b9e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:00.502555 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:00.534496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b7e32-57d9-4c86-a042-64c82cf1eab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:00.535081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b7e32-57d9-4c86-a042-64c82cf1eab9 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:00.535364 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b7e32-57d9-4c86-a042-64c82cf1eab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:00.535672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b7e32-57d9-4c86-a042-64c82cf1eab9 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:00.536292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b7e32-57d9-4c86-a042-64c82cf1eab9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:00.538194 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b7e32-57d9-4c86-a042-64c82cf1eab9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:00.560706 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1340/2677] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:35:00 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:35:00.564960 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149f090-631e-4152-b143-2633378a8628 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:00.575101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149f090-631e-4152-b143-2633378a8628 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:00.579735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c54d5-78c9-4916-b30a-b0cdece38f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:00.580335 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c54d5-78c9-4916-b30a-b0cdece38f6a admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:00.580620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c54d5-78c9-4916-b30a-b0cdece38f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:00.580896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c54d5-78c9-4916-b30a-b0cdece38f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:00.581532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c54d5-78c9-4916-b30a-b0cdece38f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:00.583103 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c54d5-78c9-4916-b30a-b0cdece38f6a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:00.593303 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1338/2678] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:35:00.596960 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b7e32-57d9-4c86-a042-64c82cf1eab9 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:00.607329 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b7e32-57d9-4c86-a042-64c82cf1eab9 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:00.612277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f378c1-9c88-483d-b419-5f2edd2ddd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:00.612879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f378c1-9c88-483d-b419-5f2edd2ddd0d admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:00.613162 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f378c1-9c88-483d-b419-5f2edd2ddd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:00.613436 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f378c1-9c88-483d-b419-5f2edd2ddd0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:00.614079 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f378c1-9c88-483d-b419-5f2edd2ddd0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:00.615717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f378c1-9c88-483d-b419-5f2edd2ddd0d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:00.626180 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1341/2679] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:35:00.630289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c54d5-78c9-4916-b30a-b0cdece38f6a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:00.640353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c54d5-78c9-4916-b30a-b0cdece38f6a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:00.645061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534648d6-5cdf-4e45-a103-4789ca523328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:00.645700 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534648d6-5cdf-4e45-a103-4789ca523328 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:00.645979 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534648d6-5cdf-4e45-a103-4789ca523328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:00.646254 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534648d6-5cdf-4e45-a103-4789ca523328 admin admin] PATH_INFO: `/v3/projects/afd3739b5c8740c3811168a0a0c730c7/users/4d520350e13e4a6aabbf7e6e94a094f1/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:00.662126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534648d6-5cdf-4e45-a103-4789ca523328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afd3739b5c8740c3811168a0a0c730c7, user_id=4d520350e13e4a6aabbf7e6e94a094f1, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:00.664751 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534648d6-5cdf-4e45-a103-4789ca523328 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:00.677992 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1339/2680] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:35:00 2022] PUT /identity/v3/projects/afd3739b5c8740c3811168a0a0c730c7/users/4d520350e13e4a6aabbf7e6e94a094f1/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:35:00.688226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42731d4-7414-4e70-8fea-e3b10a00bb0a None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:00.688703 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42731d4-7414-4e70-8fea-e3b10a00bb0a None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:00.688820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42731d4-7414-4e70-8fea-e3b10a00bb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:00.716510 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a42731d4-7414-4e70-8fea-e3b10a00bb0a None None] MFA Rules not processed for user `4d520350e13e4a6aabbf7e6e94a094f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:00.717719 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a42731d4-7414-4e70-8fea-e3b10a00bb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:00.780723 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1342/2681] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:35:00.785208 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534648d6-5cdf-4e45-a103-4789ca523328 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:00.844954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534648d6-5cdf-4e45-a103-4789ca523328 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:00.849576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e8af8-780f-41d2-aba7-2e499abdf36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:00.850166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e8af8-780f-41d2-aba7-2e499abdf36c admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:00.850442 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e8af8-780f-41d2-aba7-2e499abdf36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:00.850709 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e8af8-780f-41d2-aba7-2e499abdf36c admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:00.851330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e8af8-780f-41d2-aba7-2e499abdf36c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:00.853230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e8af8-780f-41d2-aba7-2e499abdf36c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:00.858935 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:00.881230 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1340/2682] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:35:00 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:35:00.885354 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42731d4-7414-4e70-8fea-e3b10a00bb0a None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:00.896131 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42731d4-7414-4e70-8fea-e3b10a00bb0a None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:00.951136 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541012b-8f8e-4a13-ae62-691c4b448597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:00.951763 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541012b-8f8e-4a13-ae62-691c4b448597 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:00.952048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541012b-8f8e-4a13-ae62-691c4b448597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:00.952322 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541012b-8f8e-4a13-ae62-691c4b448597 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:00.952958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541012b-8f8e-4a13-ae62-691c4b448597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:00.954977 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541012b-8f8e-4a13-ae62-691c4b448597 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:00.975918 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1343/2683] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:35:00 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:35:00.979855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e8af8-780f-41d2-aba7-2e499abdf36c admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:00.990040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e8af8-780f-41d2-aba7-2e499abdf36c admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:00.994654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797cdee-fa46-4f33-9b1a-b9656d4b96ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:00.995248 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797cdee-fa46-4f33-9b1a-b9656d4b96ea admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:00.995552 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797cdee-fa46-4f33-9b1a-b9656d4b96ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:00.995829 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797cdee-fa46-4f33-9b1a-b9656d4b96ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:00.996469 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797cdee-fa46-4f33-9b1a-b9656d4b96ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:00.998078 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797cdee-fa46-4f33-9b1a-b9656d4b96ea admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:01.008159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1341/2684] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:35:01.012836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541012b-8f8e-4a13-ae62-691c4b448597 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:01.022638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541012b-8f8e-4a13-ae62-691c4b448597 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:01.028173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed90df-034b-425b-8bd5-e00c7da886d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:01.028857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed90df-034b-425b-8bd5-e00c7da886d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:01.029139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed90df-034b-425b-8bd5-e00c7da886d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:01.029413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed90df-034b-425b-8bd5-e00c7da886d9 admin admin] PATH_INFO: `/v3/projects/9454e5487cab4e348fc00efbf69bb38c/users/0048c4f39a344180a8db938231ea2846/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:01.044169 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed90df-034b-425b-8bd5-e00c7da886d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9454e5487cab4e348fc00efbf69bb38c, user_id=0048c4f39a344180a8db938231ea2846, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:01.046798 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed90df-034b-425b-8bd5-e00c7da886d9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:01.059180 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1344/2685] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:35:01 2022] PUT /identity/v3/projects/9454e5487cab4e348fc00efbf69bb38c/users/0048c4f39a344180a8db938231ea2846/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:01.063079 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797cdee-fa46-4f33-9b1a-b9656d4b96ea admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:01.072986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797cdee-fa46-4f33-9b1a-b9656d4b96ea admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:01.128887 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b59f2-5a54-4828-ac95-4bf3632ebc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:01.129418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b59f2-5a54-4828-ac95-4bf3632ebc5c admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:01.129699 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b59f2-5a54-4828-ac95-4bf3632ebc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:01.129974 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b59f2-5a54-4828-ac95-4bf3632ebc5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:01.130605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b59f2-5a54-4828-ac95-4bf3632ebc5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:01.132277 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b59f2-5a54-4828-ac95-4bf3632ebc5c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:01.142184 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1342/2686] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:35:01.146339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed90df-034b-425b-8bd5-e00c7da886d9 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:01.151768 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:01.158978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed90df-034b-425b-8bd5-e00c7da886d9 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:01.164034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac51d8a-1814-4617-94c1-3e45482a38ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:01.165006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac51d8a-1814-4617-94c1-3e45482a38ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:01.165006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac51d8a-1814-4617-94c1-3e45482a38ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:01.165215 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac51d8a-1814-4617-94c1-3e45482a38ad admin admin] PATH_INFO: `/v3/projects/9454e5487cab4e348fc00efbf69bb38c/users/0048c4f39a344180a8db938231ea2846/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:01.167219 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac51d8a-1814-4617-94c1-3e45482a38ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=9454e5487cab4e348fc00efbf69bb38c, user_id=0048c4f39a344180a8db938231ea2846, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:01.169758 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac51d8a-1814-4617-94c1-3e45482a38ad admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:01.180047 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1345/2687] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:35:01 2022] PUT /identity/v3/projects/9454e5487cab4e348fc00efbf69bb38c/users/0048c4f39a344180a8db938231ea2846/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:01.191386 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db45c7-7aa3-42c6-9fb4-de0a00c7a92f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:01.191902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db45c7-7aa3-42c6-9fb4-de0a00c7a92f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:01.192024 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db45c7-7aa3-42c6-9fb4-de0a00c7a92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:01.220195 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-88db45c7-7aa3-42c6-9fb4-de0a00c7a92f None None] MFA Rules not processed for user `0048c4f39a344180a8db938231ea2846`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:01.221398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-88db45c7-7aa3-42c6-9fb4-de0a00c7a92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:01.287669 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1343/2688] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:35:01.298434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac51d8a-1814-4617-94c1-3e45482a38ad admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:01.300159 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5ac51d8a-1814-4617-94c1-3e45482a38ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:01.357678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac51d8a-1814-4617-94c1-3e45482a38ad admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:01.361286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414280d5-9529-440e-97b9-2c3b53b31203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:01.361877 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414280d5-9529-440e-97b9-2c3b53b31203 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:01.362162 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414280d5-9529-440e-97b9-2c3b53b31203 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:01.362435 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414280d5-9529-440e-97b9-2c3b53b31203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:01.418416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414280d5-9529-440e-97b9-2c3b53b31203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:01.419803 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414280d5-9529-440e-97b9-2c3b53b31203 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:01.428539 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1346/2689] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:35:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:35:09.122016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db45c7-7aa3-42c6-9fb4-de0a00c7a92f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:09.123449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-88db45c7-7aa3-42c6-9fb4-de0a00c7a92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:09.135252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db45c7-7aa3-42c6-9fb4-de0a00c7a92f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:09.139529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae13de4-f207-40dc-b809-26a52a83d561 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:09.140204 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae13de4-f207-40dc-b809-26a52a83d561 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:09.140482 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae13de4-f207-40dc-b809-26a52a83d561 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:09.140750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae13de4-f207-40dc-b809-26a52a83d561 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:09.149615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae13de4-f207-40dc-b809-26a52a83d561 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:09.151025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae13de4-f207-40dc-b809-26a52a83d561 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:09.161013 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1344/2690] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:35:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:35:11.424619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:19.156855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=79,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:38.053888 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414280d5-9529-440e-97b9-2c3b53b31203 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:38.116255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414280d5-9529-440e-97b9-2c3b53b31203 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:38.120824 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02d38e-320e-476d-8f32-b84d06d7ab40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:38.122133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02d38e-320e-476d-8f32-b84d06d7ab40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:38.122416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02d38e-320e-476d-8f32-b84d06d7ab40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:38.122685 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02d38e-320e-476d-8f32-b84d06d7ab40 admin admin] PATH_INFO: `/v3/users/4d520350e13e4a6aabbf7e6e94a094f1` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:38.124342 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02d38e-320e-476d-8f32-b84d06d7ab40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d520350e13e4a6aabbf7e6e94a094f1)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:38.126248 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02d38e-320e-476d-8f32-b84d06d7ab40 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:38.172707 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fa02d38e-320e-476d-8f32-b84d06d7ab40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d520350e13e4a6aabbf7e6e94a094f1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:38.179808 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fa02d38e-320e-476d-8f32-b84d06d7ab40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d520350e13e4a6aabbf7e6e94a094f1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:38.180475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fa02d38e-320e-476d-8f32-b84d06d7ab40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d520350e13e4a6aabbf7e6e94a094f1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:38.181099 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fa02d38e-320e-476d-8f32-b84d06d7ab40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d520350e13e4a6aabbf7e6e94a094f1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:38.188779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-fa02d38e-320e-476d-8f32-b84d06d7ab40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d520350e13e4a6aabbf7e6e94a094f1'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:38.206560 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1347/2691] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:35:38 2022] DELETE /identity/v3/users/4d520350e13e4a6aabbf7e6e94a094f1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:38.211162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47e90c-32ae-46d8-958d-de5f7a24a2c4 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:38.211684 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47e90c-32ae-46d8-958d-de5f7a24a2c4 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:38.211799 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47e90c-32ae-46d8-958d-de5f7a24a2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:38.230190 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-0f47e90c-32ae-46d8-958d-de5f7a24a2c4 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:38.231382 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0f47e90c-32ae-46d8-958d-de5f7a24a2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:38.290073 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1345/2692] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:35:38.299668 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02d38e-320e-476d-8f32-b84d06d7ab40 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:38.301080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-fa02d38e-320e-476d-8f32-b84d06d7ab40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:38.357713 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02d38e-320e-476d-8f32-b84d06d7ab40 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:38.361382 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bebb4-4ee0-42d0-ab11-285a42143424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:38.361984 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bebb4-4ee0-42d0-ab11-285a42143424 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:38.362267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bebb4-4ee0-42d0-ab11-285a42143424 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:38.362541 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bebb4-4ee0-42d0-ab11-285a42143424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:38.371455 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bebb4-4ee0-42d0-ab11-285a42143424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:38.372911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bebb4-4ee0-42d0-ab11-285a42143424 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:38.382701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1348/2693] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:35:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:35:38.642590 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47e90c-32ae-46d8-958d-de5f7a24a2c4 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:38.644154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0f47e90c-32ae-46d8-958d-de5f7a24a2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:38.656322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47e90c-32ae-46d8-958d-de5f7a24a2c4 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:38.661286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:38.662011 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:38.662368 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:38.662648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219 admin admin] PATH_INFO: `/v3/projects/afd3739b5c8740c3811168a0a0c730c7` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:38.663811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afd3739b5c8740c3811168a0a0c730c7)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:38.665782 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:38.712195 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afd3739b5c8740c3811168a0a0c730c7', 'request_id': 'req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:38.719223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afd3739b5c8740c3811168a0a0c730c7', 'request_id': 'req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:38.725461 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afd3739b5c8740c3811168a0a0c730c7', 'request_id': 'req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:38.736676 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219 admin admin] The token cache is being invalidate because project afd3739b5c8740c3811168a0a0c730c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:35:38.736994 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:38.737629 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:38.739102 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1346/2694] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:35:38 2022] DELETE /identity/v3/projects/afd3739b5c8740c3811168a0a0c730c7 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:38.743167 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bebb4-4ee0-42d0-ab11-285a42143424 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:38.753346 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bebb4-4ee0-42d0-ab11-285a42143424 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:38.770956 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:38.813423 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2aac9-2c53-48d4-9ff1-6aade7aaf7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:38.814055 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2aac9-2c53-48d4-9ff1-6aade7aaf7f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:38.814333 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2aac9-2c53-48d4-9ff1-6aade7aaf7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:38.814601 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2aac9-2c53-48d4-9ff1-6aade7aaf7f3 admin admin] PATH_INFO: `/v3/users/0048c4f39a344180a8db938231ea2846` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:38.815741 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2aac9-2c53-48d4-9ff1-6aade7aaf7f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0048c4f39a344180a8db938231ea2846)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:38.817703 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2aac9-2c53-48d4-9ff1-6aade7aaf7f3 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:38.862983 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9dd2aac9-2c53-48d4-9ff1-6aade7aaf7f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0048c4f39a344180a8db938231ea2846'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:38.869793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9dd2aac9-2c53-48d4-9ff1-6aade7aaf7f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0048c4f39a344180a8db938231ea2846'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:38.870451 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9dd2aac9-2c53-48d4-9ff1-6aade7aaf7f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0048c4f39a344180a8db938231ea2846'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:38.871075 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9dd2aac9-2c53-48d4-9ff1-6aade7aaf7f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0048c4f39a344180a8db938231ea2846'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:38.878261 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-9dd2aac9-2c53-48d4-9ff1-6aade7aaf7f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0048c4f39a344180a8db938231ea2846'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:38.895199 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1349/2695] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:35:38 2022] DELETE /identity/v3/users/0048c4f39a344180a8db938231ea2846 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:38.957588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:38.959256 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:39.018003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b4ef7-eee2-4eb4-8a4c-cd3844aa5219 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:39.057559 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:39.071492 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137ed10-be35-48c8-934c-50a6b8fbd41b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:39.072127 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137ed10-be35-48c8-934c-50a6b8fbd41b service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:39.072405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137ed10-be35-48c8-934c-50a6b8fbd41b service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:39.072677 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137ed10-be35-48c8-934c-50a6b8fbd41b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:39.074398 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4137ed10-be35-48c8-934c-50a6b8fbd41b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:39.161444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137ed10-be35-48c8-934c-50a6b8fbd41b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:39.163129 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137ed10-be35-48c8-934c-50a6b8fbd41b service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:39.171609 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2aac9-2c53-48d4-9ff1-6aade7aaf7f3 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:39.173209 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-9dd2aac9-2c53-48d4-9ff1-6aade7aaf7f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:39.184080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1347/2696] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:35:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:35:39.241831 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2aac9-2c53-48d4-9ff1-6aade7aaf7f3 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:39.252936 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac3490-e334-4b55-849b-959adadd831c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:39.253601 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac3490-e334-4b55-849b-959adadd831c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:39.253887 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac3490-e334-4b55-849b-959adadd831c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:39.254217 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac3490-e334-4b55-849b-959adadd831c admin admin] PATH_INFO: `/v3/projects/9454e5487cab4e348fc00efbf69bb38c` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:39.255409 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac3490-e334-4b55-849b-959adadd831c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9454e5487cab4e348fc00efbf69bb38c)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:39.257343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac3490-e334-4b55-849b-959adadd831c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:39.302717 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:39.307088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8fac3490-e334-4b55-849b-959adadd831c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9454e5487cab4e348fc00efbf69bb38c', 'request_id': 'req-8fac3490-e334-4b55-849b-959adadd831c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:39.315576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8fac3490-e334-4b55-849b-959adadd831c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9454e5487cab4e348fc00efbf69bb38c', 'request_id': 'req-8fac3490-e334-4b55-849b-959adadd831c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:39.322420 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8fac3490-e334-4b55-849b-959adadd831c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9454e5487cab4e348fc00efbf69bb38c', 'request_id': 'req-8fac3490-e334-4b55-849b-959adadd831c'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:39.335238 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8fac3490-e334-4b55-849b-959adadd831c admin admin] The token cache is being invalidate because project 9454e5487cab4e348fc00efbf69bb38c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:35:39.335674 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8fac3490-e334-4b55-849b-959adadd831c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:39.336494 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8fac3490-e334-4b55-849b-959adadd831c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:39.338166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1350/2697] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:35:39 2022] DELETE /identity/v3/projects/9454e5487cab4e348fc00efbf69bb38c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:35:39.345044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567ec2d-c875-4eda-b9e2-baff6a8b743e None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:39.345626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567ec2d-c875-4eda-b9e2-baff6a8b743e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:39.345894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567ec2d-c875-4eda-b9e2-baff6a8b743e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:39.365522 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e567ec2d-c875-4eda-b9e2-baff6a8b743e None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:39.366723 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e567ec2d-c875-4eda-b9e2-baff6a8b743e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:39.425980 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1348/2698] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:35:39.433812 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f3c70-f153-4b8d-8e47-cccbe93a1e72 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:39.434202 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f3c70-f153-4b8d-8e47-cccbe93a1e72 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:39.434544 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f3c70-f153-4b8d-8e47-cccbe93a1e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:39.454835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6e0f3c70-f153-4b8d-8e47-cccbe93a1e72 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:39.456190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6e0f3c70-f153-4b8d-8e47-cccbe93a1e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:39.464318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1351/2699] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:35:39.469020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567ec2d-c875-4eda-b9e2-baff6a8b743e None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:39.478614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567ec2d-c875-4eda-b9e2-baff6a8b743e None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:39.483209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4fe6d-1e54-472c-a4bf-03ee9d6bba87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:39.483870 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4fe6d-1e54-472c-a4bf-03ee9d6bba87 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:39.484153 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4fe6d-1e54-472c-a4bf-03ee9d6bba87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:39.484429 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4fe6d-1e54-472c-a4bf-03ee9d6bba87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:39.485070 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4fe6d-1e54-472c-a4bf-03ee9d6bba87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:35:39.485379 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4fe6d-1e54-472c-a4bf-03ee9d6bba87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:39.486969 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4fe6d-1e54-472c-a4bf-03ee9d6bba87 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:39.498393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1349/2700] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:35:39.502747 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f3c70-f153-4b8d-8e47-cccbe93a1e72 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:39.511958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f3c70-f153-4b8d-8e47-cccbe93a1e72 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:39.517115 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d38605-41ef-48b4-84f9-b262fd398963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:39.517706 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d38605-41ef-48b4-84f9-b262fd398963 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:39.517983 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d38605-41ef-48b4-84f9-b262fd398963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:39.518252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d38605-41ef-48b4-84f9-b262fd398963 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:39.518870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d38605-41ef-48b4-84f9-b262fd398963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:39.520787 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d38605-41ef-48b4-84f9-b262fd398963 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:39.550288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1352/2701] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:35:39 2022] POST /identity/v3/projects => generated 409 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:35:39.554085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4fe6d-1e54-472c-a4bf-03ee9d6bba87 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:39.564755 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4fe6d-1e54-472c-a4bf-03ee9d6bba87 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:39.614481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:39.625244 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5d58b-a4d3-449b-bb50-ca0e92df89d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:39.625860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5d58b-a4d3-449b-bb50-ca0e92df89d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:39.626136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5d58b-a4d3-449b-bb50-ca0e92df89d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:39.626407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5d58b-a4d3-449b-bb50-ca0e92df89d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:39.627965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5d58b-a4d3-449b-bb50-ca0e92df89d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:39.630301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5d58b-a4d3-449b-bb50-ca0e92df89d7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:39.655226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1350/2702] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:35:39 2022] POST /identity/v3/users => generated 305 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:35:39.659555 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d38605-41ef-48b4-84f9-b262fd398963 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:39.670178 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d38605-41ef-48b4-84f9-b262fd398963 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:39.674972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b516c2-d937-462c-bc2c-219f91bc5935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:39.675598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b516c2-d937-462c-bc2c-219f91bc5935 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:39.675886 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b516c2-d937-462c-bc2c-219f91bc5935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:39.676163 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b516c2-d937-462c-bc2c-219f91bc5935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:39.676835 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b516c2-d937-462c-bc2c-219f91bc5935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:39.678426 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b516c2-d937-462c-bc2c-219f91bc5935 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:39.689421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1353/2703] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:35:39.693354 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5d58b-a4d3-449b-bb50-ca0e92df89d7 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:39.704502 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5d58b-a4d3-449b-bb50-ca0e92df89d7 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:39.709580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f04c88-b774-498f-9f0b-288aab7ac052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:39.710167 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f04c88-b774-498f-9f0b-288aab7ac052 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:39.710449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f04c88-b774-498f-9f0b-288aab7ac052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:39.710721 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f04c88-b774-498f-9f0b-288aab7ac052 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:39.711367 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f04c88-b774-498f-9f0b-288aab7ac052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:39.713050 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f04c88-b774-498f-9f0b-288aab7ac052 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:39.723943 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1351/2704] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:35:39.728041 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b516c2-d937-462c-bc2c-219f91bc5935 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:39.737857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b516c2-d937-462c-bc2c-219f91bc5935 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:39.742653 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2c9e0-3e63-4e57-b75b-71606faf2f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:39.743303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2c9e0-3e63-4e57-b75b-71606faf2f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:39.743614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2c9e0-3e63-4e57-b75b-71606faf2f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:39.743892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2c9e0-3e63-4e57-b75b-71606faf2f74 admin admin] PATH_INFO: `/v3/projects/76af68a5acb8426fbba5f5b3a105c08b/users/7912cc98c22146bc9478372b3e1c7123/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:39.760093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2c9e0-3e63-4e57-b75b-71606faf2f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76af68a5acb8426fbba5f5b3a105c08b, user_id=7912cc98c22146bc9478372b3e1c7123, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:39.762726 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2c9e0-3e63-4e57-b75b-71606faf2f74 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:39.776406 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1354/2705] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:35:39 2022] PUT /identity/v3/projects/76af68a5acb8426fbba5f5b3a105c08b/users/7912cc98c22146bc9478372b3e1c7123/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:39.788085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81807cb-bb6d-4da3-9317-1fec76f6b51b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:39.788612 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81807cb-bb6d-4da3-9317-1fec76f6b51b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:39.788979 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81807cb-bb6d-4da3-9317-1fec76f6b51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:39.818241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f81807cb-bb6d-4da3-9317-1fec76f6b51b None None] MFA Rules not processed for user `7912cc98c22146bc9478372b3e1c7123`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:39.819617 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f81807cb-bb6d-4da3-9317-1fec76f6b51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:39.885251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1352/2706] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:39 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:35:39.889883 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2c9e0-3e63-4e57-b75b-71606faf2f74 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:39.950741 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2c9e0-3e63-4e57-b75b-71606faf2f74 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:39.956470 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435d675-c2dd-4861-a740-62452dd83c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:39.957112 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435d675-c2dd-4861-a740-62452dd83c45 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:39.957391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435d675-c2dd-4861-a740-62452dd83c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:39.957660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435d675-c2dd-4861-a740-62452dd83c45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:39.958317 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435d675-c2dd-4861-a740-62452dd83c45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:39.960252 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435d675-c2dd-4861-a740-62452dd83c45 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:39.987576 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1355/2707] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:35:39 2022] POST /identity/v3/projects => generated 407 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:35:39.991612 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81807cb-bb6d-4da3-9317-1fec76f6b51b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:40.002746 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81807cb-bb6d-4da3-9317-1fec76f6b51b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:40.066085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1cd75-27c2-4dec-800a-501a0da3a6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:40.066786 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1cd75-27c2-4dec-800a-501a0da3a6f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:40.067067 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1cd75-27c2-4dec-800a-501a0da3a6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:40.067339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1cd75-27c2-4dec-800a-501a0da3a6f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:40.068037 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1cd75-27c2-4dec-800a-501a0da3a6f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:40.069958 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1cd75-27c2-4dec-800a-501a0da3a6f8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:40.081665 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435d675-c2dd-4861-a740-62452dd83c45 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:40.083799 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-f435d675-c2dd-4861-a740-62452dd83c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:40.103978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1353/2708] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:35:39 2022] POST /identity/v3/users => generated 310 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:35:40.106642 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1cd75-27c2-4dec-800a-501a0da3a6f8 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:40.126419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1cd75-27c2-4dec-800a-501a0da3a6f8 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:40.132906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49648263-570a-4e6c-87be-2bf228f8d411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:40.133841 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49648263-570a-4e6c-87be-2bf228f8d411 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:40.134226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49648263-570a-4e6c-87be-2bf228f8d411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:40.135139 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49648263-570a-4e6c-87be-2bf228f8d411 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:40.136649 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49648263-570a-4e6c-87be-2bf228f8d411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:40.138641 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49648263-570a-4e6c-87be-2bf228f8d411 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:40.154455 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1354/2709] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:35:40.156894 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49648263-570a-4e6c-87be-2bf228f8d411 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:40.163006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:40.177054 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49648263-570a-4e6c-87be-2bf228f8d411 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:40.182517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08469a19-8429-4c9d-b247-0467a03ffabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:40.183189 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08469a19-8429-4c9d-b247-0467a03ffabd admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:40.183475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08469a19-8429-4c9d-b247-0467a03ffabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:40.183803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08469a19-8429-4c9d-b247-0467a03ffabd admin admin] PATH_INFO: `/v3/projects/bbc583c7be2e43ceae56ffc7979d60ba/users/3dbb46d059e84d6ca0450e0f60a1baab/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:40.197066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435d675-c2dd-4861-a740-62452dd83c45 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:40.202660 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08469a19-8429-4c9d-b247-0467a03ffabd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbc583c7be2e43ceae56ffc7979d60ba, user_id=3dbb46d059e84d6ca0450e0f60a1baab, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:40.204269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3ff2f-62d4-4cbc-b0a2-4d37c43da0f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:40.205148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3ff2f-62d4-4cbc-b0a2-4d37c43da0f1 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:40.205527 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3ff2f-62d4-4cbc-b0a2-4d37c43da0f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:40.205657 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08469a19-8429-4c9d-b247-0467a03ffabd admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:40.206084 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3ff2f-62d4-4cbc-b0a2-4d37c43da0f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:40.208308 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-3be3ff2f-62d4-4cbc-b0a2-4d37c43da0f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:40.223596 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3ff2f-62d4-4cbc-b0a2-4d37c43da0f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:40.225497 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3ff2f-62d4-4cbc-b0a2-4d37c43da0f1 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:40.237579 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1355/2710] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:35:40 2022] PUT /identity/v3/projects/bbc583c7be2e43ceae56ffc7979d60ba/users/3dbb46d059e84d6ca0450e0f60a1baab/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:40.240885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1356/2711] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:35:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:35:40.256632 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08469a19-8429-4c9d-b247-0467a03ffabd admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:40.289298 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:40.341873 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08469a19-8429-4c9d-b247-0467a03ffabd admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:40.348097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d3c58-2962-4e4e-859f-9f9f9d2d7220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:40.348804 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d3c58-2962-4e4e-859f-9f9f9d2d7220 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:40.349044 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d3c58-2962-4e4e-859f-9f9f9d2d7220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:40.349324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d3c58-2962-4e4e-859f-9f9f9d2d7220 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:40.349978 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d3c58-2962-4e4e-859f-9f9f9d2d7220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:40.352299 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d3c58-2962-4e4e-859f-9f9f9d2d7220 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:40.364010 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1356/2712] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:35:40.367951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3ff2f-62d4-4cbc-b0a2-4d37c43da0f1 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:40.378258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3ff2f-62d4-4cbc-b0a2-4d37c43da0f1 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:40.382969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0085210-c0c4-4842-81a3-e4562ef7a902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:40.383670 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0085210-c0c4-4842-81a3-e4562ef7a902 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:40.383929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0085210-c0c4-4842-81a3-e4562ef7a902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:40.384201 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0085210-c0c4-4842-81a3-e4562ef7a902 admin admin] PATH_INFO: `/v3/projects/bbc583c7be2e43ceae56ffc7979d60ba/users/3dbb46d059e84d6ca0450e0f60a1baab/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:40.386153 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0085210-c0c4-4842-81a3-e4562ef7a902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbc583c7be2e43ceae56ffc7979d60ba, user_id=3dbb46d059e84d6ca0450e0f60a1baab, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:40.388944 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0085210-c0c4-4842-81a3-e4562ef7a902 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:40.400211 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1357/2713] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:35:40 2022] PUT /identity/v3/projects/bbc583c7be2e43ceae56ffc7979d60ba/users/3dbb46d059e84d6ca0450e0f60a1baab/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:40.412163 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b80f84-9405-4334-a286-5e69f7302685 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:40.412707 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b80f84-9405-4334-a286-5e69f7302685 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:40.413584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b80f84-9405-4334-a286-5e69f7302685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:40.446698 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-31b80f84-9405-4334-a286-5e69f7302685 None None] MFA Rules not processed for user `3dbb46d059e84d6ca0450e0f60a1baab`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:40.447999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-31b80f84-9405-4334-a286-5e69f7302685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:40.516402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1357/2714] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:40 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:35:40.527570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0085210-c0c4-4842-81a3-e4562ef7a902 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:40.529326 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-e0085210-c0c4-4842-81a3-e4562ef7a902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:40.588515 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0085210-c0c4-4842-81a3-e4562ef7a902 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:40.592950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed966363-8a6f-4357-9e48-58da024426c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:40.593670 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed966363-8a6f-4357-9e48-58da024426c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:40.593959 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed966363-8a6f-4357-9e48-58da024426c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:40.594236 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed966363-8a6f-4357-9e48-58da024426c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:40.603487 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed966363-8a6f-4357-9e48-58da024426c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:40.604943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed966363-8a6f-4357-9e48-58da024426c3 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:40.615166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1358/2715] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:35:40 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:35:40.778862 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b80f84-9405-4334-a286-5e69f7302685 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:40.789936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b80f84-9405-4334-a286-5e69f7302685 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:40.793573 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96505d-4781-4968-8e5e-a398db1fe8b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:40.794194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96505d-4781-4968-8e5e-a398db1fe8b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:40.794475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96505d-4781-4968-8e5e-a398db1fe8b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:40.794749 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96505d-4781-4968-8e5e-a398db1fe8b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:40.855608 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96505d-4781-4968-8e5e-a398db1fe8b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:40.857041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96505d-4781-4968-8e5e-a398db1fe8b2 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:40.866216 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1358/2716] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:35:40 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:35:43.383753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed966363-8a6f-4357-9e48-58da024426c3 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:43.455101 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed966363-8a6f-4357-9e48-58da024426c3 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:43.460591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f8938-4e18-431c-aa62-5b200963d071 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:43.461191 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f8938-4e18-431c-aa62-5b200963d071 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:43.461469 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f8938-4e18-431c-aa62-5b200963d071 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:43.461742 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f8938-4e18-431c-aa62-5b200963d071 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:43.463529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c45f8938-4e18-431c-aa62-5b200963d071 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:43.475340 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f8938-4e18-431c-aa62-5b200963d071 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:43.476932 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f8938-4e18-431c-aa62-5b200963d071 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:43.487403 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1359/2717] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:35:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:35:47.887563 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96505d-4781-4968-8e5e-a398db1fe8b2 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:47.889533 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-dc96505d-4781-4968-8e5e-a398db1fe8b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:47.928383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=100 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:47.958800 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96505d-4781-4968-8e5e-a398db1fe8b2 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:47.963784 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd355854-e1e4-454e-8c82-484ffc9319b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:47.964400 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd355854-e1e4-454e-8c82-484ffc9319b9 service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:47.964674 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd355854-e1e4-454e-8c82-484ffc9319b9 service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:47.964942 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd355854-e1e4-454e-8c82-484ffc9319b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:47.966615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-dd355854-e1e4-454e-8c82-484ffc9319b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:47.977833 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd355854-e1e4-454e-8c82-484ffc9319b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:47.979407 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd355854-e1e4-454e-8c82-484ffc9319b9 service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:47.989321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1359/2718] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:35:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:35:51.275617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f8938-4e18-431c-aa62-5b200963d071 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:51.325345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:51.336338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f8938-4e18-431c-aa62-5b200963d071 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:51.340782 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63295480-a01e-4056-a4d6-42213c6ea1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:51.341398 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63295480-a01e-4056-a4d6-42213c6ea1f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:51.341677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63295480-a01e-4056-a4d6-42213c6ea1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:51.341971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63295480-a01e-4056-a4d6-42213c6ea1f0 admin admin] PATH_INFO: `/v3/users/7912cc98c22146bc9478372b3e1c7123` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:51.343155 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63295480-a01e-4056-a4d6-42213c6ea1f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7912cc98c22146bc9478372b3e1c7123)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:51.345091 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63295480-a01e-4056-a4d6-42213c6ea1f0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:51.390500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-63295480-a01e-4056-a4d6-42213c6ea1f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7912cc98c22146bc9478372b3e1c7123'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:51.397260 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-63295480-a01e-4056-a4d6-42213c6ea1f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7912cc98c22146bc9478372b3e1c7123'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:51.397928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-63295480-a01e-4056-a4d6-42213c6ea1f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7912cc98c22146bc9478372b3e1c7123'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:51.398544 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-63295480-a01e-4056-a4d6-42213c6ea1f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7912cc98c22146bc9478372b3e1c7123'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:51.408996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-63295480-a01e-4056-a4d6-42213c6ea1f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7912cc98c22146bc9478372b3e1c7123'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:51.428105 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1360/2719] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:35:51 2022] DELETE /identity/v3/users/7912cc98c22146bc9478372b3e1c7123 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:35:51.432751 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2912edf-e7c2-40e9-8531-a892895eca70 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:51.433568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2912edf-e7c2-40e9-8531-a892895eca70 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:51.433568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2912edf-e7c2-40e9-8531-a892895eca70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:51.452847 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e2912edf-e7c2-40e9-8531-a892895eca70 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:51.454160 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e2912edf-e7c2-40e9-8531-a892895eca70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:51.513031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1360/2720] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:35:51.523800 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63295480-a01e-4056-a4d6-42213c6ea1f0 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:51.525192 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-63295480-a01e-4056-a4d6-42213c6ea1f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:51.584210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63295480-a01e-4056-a4d6-42213c6ea1f0 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:51.587776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd8acc-7c1e-4484-87c4-bdd70d9dc440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:51.588377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd8acc-7c1e-4484-87c4-bdd70d9dc440 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:51.588657 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd8acc-7c1e-4484-87c4-bdd70d9dc440 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:51.589016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd8acc-7c1e-4484-87c4-bdd70d9dc440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:51.598781 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd8acc-7c1e-4484-87c4-bdd70d9dc440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:51.600281 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd8acc-7c1e-4484-87c4-bdd70d9dc440 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:51.610834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1361/2721] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:35:51.869197 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2912edf-e7c2-40e9-8531-a892895eca70 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:51.870618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e2912edf-e7c2-40e9-8531-a892895eca70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:51.882816 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2912edf-e7c2-40e9-8531-a892895eca70 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:51.887498 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421dfee-9ced-49c0-b688-a8754672d286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:51.888268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421dfee-9ced-49c0-b688-a8754672d286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:51.888553 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421dfee-9ced-49c0-b688-a8754672d286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:51.888893 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421dfee-9ced-49c0-b688-a8754672d286 admin admin] PATH_INFO: `/v3/projects/76af68a5acb8426fbba5f5b3a105c08b` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:51.890047 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421dfee-9ced-49c0-b688-a8754672d286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76af68a5acb8426fbba5f5b3a105c08b)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:51.892114 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421dfee-9ced-49c0-b688-a8754672d286 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:51.947449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0421dfee-9ced-49c0-b688-a8754672d286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76af68a5acb8426fbba5f5b3a105c08b', 'request_id': 'req-0421dfee-9ced-49c0-b688-a8754672d286'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:51.955983 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0421dfee-9ced-49c0-b688-a8754672d286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76af68a5acb8426fbba5f5b3a105c08b', 'request_id': 'req-0421dfee-9ced-49c0-b688-a8754672d286'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:51.962735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0421dfee-9ced-49c0-b688-a8754672d286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76af68a5acb8426fbba5f5b3a105c08b', 'request_id': 'req-0421dfee-9ced-49c0-b688-a8754672d286'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:51.973706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0421dfee-9ced-49c0-b688-a8754672d286 admin admin] The token cache is being invalidate because project 76af68a5acb8426fbba5f5b3a105c08b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:35:51.973990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0421dfee-9ced-49c0-b688-a8754672d286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:51.974640 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-0421dfee-9ced-49c0-b688-a8754672d286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:51.976174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1361/2722] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:35:51 2022] DELETE /identity/v3/projects/76af68a5acb8426fbba5f5b3a105c08b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:51.980431 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd8acc-7c1e-4484-87c4-bdd70d9dc440 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:51.989817 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd8acc-7c1e-4484-87c4-bdd70d9dc440 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:52.049356 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ca96a-50cd-40fe-b36b-7d59f2828a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:52.050301 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ca96a-50cd-40fe-b36b-7d59f2828a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:52.050578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ca96a-50cd-40fe-b36b-7d59f2828a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:52.050870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ca96a-50cd-40fe-b36b-7d59f2828a71 admin admin] PATH_INFO: `/v3/users/3dbb46d059e84d6ca0450e0f60a1baab` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:52.052045 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ca96a-50cd-40fe-b36b-7d59f2828a71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dbb46d059e84d6ca0450e0f60a1baab)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:52.053948 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ca96a-50cd-40fe-b36b-7d59f2828a71 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:52.070071 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:52.102393 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f16ca96a-50cd-40fe-b36b-7d59f2828a71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dbb46d059e84d6ca0450e0f60a1baab'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:52.109732 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f16ca96a-50cd-40fe-b36b-7d59f2828a71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dbb46d059e84d6ca0450e0f60a1baab'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:52.110408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f16ca96a-50cd-40fe-b36b-7d59f2828a71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dbb46d059e84d6ca0450e0f60a1baab'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:52.111025 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f16ca96a-50cd-40fe-b36b-7d59f2828a71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dbb46d059e84d6ca0450e0f60a1baab'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:52.120781 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-f16ca96a-50cd-40fe-b36b-7d59f2828a71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dbb46d059e84d6ca0450e0f60a1baab'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:52.141838 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1362/2723] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:35:51 2022] DELETE /identity/v3/users/3dbb46d059e84d6ca0450e0f60a1baab => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:52.418433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421dfee-9ced-49c0-b688-a8754672d286 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:52.420159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0421dfee-9ced-49c0-b688-a8754672d286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:52.433437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421dfee-9ced-49c0-b688-a8754672d286 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:52.487958 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:52.501750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39572507-cc4f-4524-a760-21ccbc0e07ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:52.502467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39572507-cc4f-4524-a760-21ccbc0e07ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:52.502814 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39572507-cc4f-4524-a760-21ccbc0e07ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:52.503169 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39572507-cc4f-4524-a760-21ccbc0e07ec admin admin] PATH_INFO: `/v3/projects/bbc583c7be2e43ceae56ffc7979d60ba` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:52.504551 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39572507-cc4f-4524-a760-21ccbc0e07ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbc583c7be2e43ceae56ffc7979d60ba)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:52.506550 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39572507-cc4f-4524-a760-21ccbc0e07ec admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:52.552546 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-39572507-cc4f-4524-a760-21ccbc0e07ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbc583c7be2e43ceae56ffc7979d60ba', 'request_id': 'req-39572507-cc4f-4524-a760-21ccbc0e07ec'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:52.559811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-39572507-cc4f-4524-a760-21ccbc0e07ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbc583c7be2e43ceae56ffc7979d60ba', 'request_id': 'req-39572507-cc4f-4524-a760-21ccbc0e07ec'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:52.567059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-39572507-cc4f-4524-a760-21ccbc0e07ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbc583c7be2e43ceae56ffc7979d60ba', 'request_id': 'req-39572507-cc4f-4524-a760-21ccbc0e07ec'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:52.578377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-39572507-cc4f-4524-a760-21ccbc0e07ec admin admin] The token cache is being invalidate because project bbc583c7be2e43ceae56ffc7979d60ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:35:52.578669 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-39572507-cc4f-4524-a760-21ccbc0e07ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:52.579333 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-39572507-cc4f-4524-a760-21ccbc0e07ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:52.581074 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1362/2724] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:35:52 2022] DELETE /identity/v3/projects/bbc583c7be2e43ceae56ffc7979d60ba => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:52.585704 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142099c4-85c9-42b5-a548-fb23afee6881 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:52.586197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142099c4-85c9-42b5-a548-fb23afee6881 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:52.586299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142099c4-85c9-42b5-a548-fb23afee6881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:52.605584 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-142099c4-85c9-42b5-a548-fb23afee6881 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:52.606795 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-142099c4-85c9-42b5-a548-fb23afee6881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:52.666994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1363/2725] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:35:52.735932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d165bf-ea24-4c8c-a748-073acc42708b None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:52.736294 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d165bf-ea24-4c8c-a748-073acc42708b None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:52.736795 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d165bf-ea24-4c8c-a748-073acc42708b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:52.771193 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-35d165bf-ea24-4c8c-a748-073acc42708b None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:52.772473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-35d165bf-ea24-4c8c-a748-073acc42708b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:52.789844 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142099c4-85c9-42b5-a548-fb23afee6881 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:52.800739 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1363/2726] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:35:52.801202 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142099c4-85c9-42b5-a548-fb23afee6881 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:52.806559 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b0158-71a5-420b-9ee5-5830bda52471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:52.807235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b0158-71a5-420b-9ee5-5830bda52471 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:52.807607 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b0158-71a5-420b-9ee5-5830bda52471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:52.807943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b0158-71a5-420b-9ee5-5830bda52471 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:52.808661 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b0158-71a5-420b-9ee5-5830bda52471 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:35:52.809035 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b0158-71a5-420b-9ee5-5830bda52471 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:52.812418 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b0158-71a5-420b-9ee5-5830bda52471 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:52.829292 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d165bf-ea24-4c8c-a748-073acc42708b None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:52.835643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1364/2727] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 12:35:52.841767 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d165bf-ea24-4c8c-a748-073acc42708b None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:52.847747 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab2d00-f3c0-42ba-b039-cc08b239dec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:52.848361 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab2d00-f3c0-42ba-b039-cc08b239dec0 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:52.848631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab2d00-f3c0-42ba-b039-cc08b239dec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:52.848897 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab2d00-f3c0-42ba-b039-cc08b239dec0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:52.849528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab2d00-f3c0-42ba-b039-cc08b239dec0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:52.851457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab2d00-f3c0-42ba-b039-cc08b239dec0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:52.889778 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1364/2728] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:35:52 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:35:52.894421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b0158-71a5-420b-9ee5-5830bda52471 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:52.905674 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b0158-71a5-420b-9ee5-5830bda52471 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:52.971595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa9157-4609-42a5-94f6-3acc21680e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:52.971595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa9157-4609-42a5-94f6-3acc21680e58 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:52.971595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa9157-4609-42a5-94f6-3acc21680e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:52.971969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa9157-4609-42a5-94f6-3acc21680e58 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:52.972480 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa9157-4609-42a5-94f6-3acc21680e58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:52.975349 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa9157-4609-42a5-94f6-3acc21680e58 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:52.998029 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1365/2729] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:35:52 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:35:53.002645 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab2d00-f3c0-42ba-b039-cc08b239dec0 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:53.013517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab2d00-f3c0-42ba-b039-cc08b239dec0 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:53.021565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9b04c-eb6f-4164-bf83-d4ef0ee65e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:53.021788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9b04c-eb6f-4164-bf83-d4ef0ee65e54 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:53.021901 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9b04c-eb6f-4164-bf83-d4ef0ee65e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:53.022143 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9b04c-eb6f-4164-bf83-d4ef0ee65e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:53.022797 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9b04c-eb6f-4164-bf83-d4ef0ee65e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:53.024422 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9b04c-eb6f-4164-bf83-d4ef0ee65e54 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:53.035750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1365/2730] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:35:53.039698 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa9157-4609-42a5-94f6-3acc21680e58 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:53.051226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa9157-4609-42a5-94f6-3acc21680e58 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:53.057051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f0277-e7f3-4f3c-90c2-6931e7b97828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:53.057800 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f0277-e7f3-4f3c-90c2-6931e7b97828 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:53.058184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f0277-e7f3-4f3c-90c2-6931e7b97828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:53.058575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f0277-e7f3-4f3c-90c2-6931e7b97828 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:53.059535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f0277-e7f3-4f3c-90c2-6931e7b97828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:53.061969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f0277-e7f3-4f3c-90c2-6931e7b97828 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:53.068237 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:53.077784 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1366/2731] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:35:53.080142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9b04c-eb6f-4164-bf83-d4ef0ee65e54 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:53.090900 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9b04c-eb6f-4164-bf83-d4ef0ee65e54 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:53.096765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018689a-3a42-48ce-b5f7-1a250421bcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:53.097469 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018689a-3a42-48ce-b5f7-1a250421bcf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:53.097738 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018689a-3a42-48ce-b5f7-1a250421bcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:53.098150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018689a-3a42-48ce-b5f7-1a250421bcf1 admin admin] PATH_INFO: `/v3/projects/a3812d2edd4e4b5989a1d2834d74c7e0/users/994aaea47f5040f2b63da909e135a16b/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:53.117301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018689a-3a42-48ce-b5f7-1a250421bcf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3812d2edd4e4b5989a1d2834d74c7e0, user_id=994aaea47f5040f2b63da909e135a16b, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:53.119809 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018689a-3a42-48ce-b5f7-1a250421bcf1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:53.138104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1366/2732] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:35:53 2022] PUT /identity/v3/projects/a3812d2edd4e4b5989a1d2834d74c7e0/users/994aaea47f5040f2b63da909e135a16b/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:53.152536 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24679d-edf3-4081-963f-889e93fecdec None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:53.152849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24679d-edf3-4081-963f-889e93fecdec None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:53.153218 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24679d-edf3-4081-963f-889e93fecdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:53.183753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-da24679d-edf3-4081-963f-889e93fecdec None None] MFA Rules not processed for user `994aaea47f5040f2b63da909e135a16b`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:53.185069 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-da24679d-edf3-4081-963f-889e93fecdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:53.262649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1367/2733] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:35:53.275135 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018689a-3a42-48ce-b5f7-1a250421bcf1 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:53.276854 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5018689a-3a42-48ce-b5f7-1a250421bcf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:53.335116 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018689a-3a42-48ce-b5f7-1a250421bcf1 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:53.338768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5ebee-a270-4b7f-bcd9-db0eedfddfa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:53.339391 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5ebee-a270-4b7f-bcd9-db0eedfddfa6 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:53.339719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5ebee-a270-4b7f-bcd9-db0eedfddfa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:53.339991 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5ebee-a270-4b7f-bcd9-db0eedfddfa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:53.349300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5ebee-a270-4b7f-bcd9-db0eedfddfa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:53.350725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5ebee-a270-4b7f-bcd9-db0eedfddfa6 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:53.360385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1367/2734] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:35:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:35:53.383750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24679d-edf3-4081-963f-889e93fecdec None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:53.446902 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24679d-edf3-4081-963f-889e93fecdec None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:53.452081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178db52-0255-459b-8c4e-3816a4c8f771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:53.452718 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178db52-0255-459b-8c4e-3816a4c8f771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:53.452995 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178db52-0255-459b-8c4e-3816a4c8f771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:53.453264 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178db52-0255-459b-8c4e-3816a4c8f771 admin admin] PATH_INFO: `/v3/users/994aaea47f5040f2b63da909e135a16b` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:53.454422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178db52-0255-459b-8c4e-3816a4c8f771 admin admin] RBAC: Authorizing `identity:delete_user(user_id=994aaea47f5040f2b63da909e135a16b)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:53.456367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178db52-0255-459b-8c4e-3816a4c8f771 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:53.506485 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2178db52-0255-459b-8c4e-3816a4c8f771 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '994aaea47f5040f2b63da909e135a16b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:53.513652 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2178db52-0255-459b-8c4e-3816a4c8f771 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '994aaea47f5040f2b63da909e135a16b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:53.514937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2178db52-0255-459b-8c4e-3816a4c8f771 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '994aaea47f5040f2b63da909e135a16b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:53.515587 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2178db52-0255-459b-8c4e-3816a4c8f771 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '994aaea47f5040f2b63da909e135a16b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:53.523722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2178db52-0255-459b-8c4e-3816a4c8f771 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '994aaea47f5040f2b63da909e135a16b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:53.542277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1368/2735] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:35:53 2022] DELETE /identity/v3/users/994aaea47f5040f2b63da909e135a16b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:53.549417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a7957-1e4b-4141-a321-fd902abfeb28 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:53.549874 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a7957-1e4b-4141-a321-fd902abfeb28 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:53.550152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a7957-1e4b-4141-a321-fd902abfeb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:53.574502 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-786a7957-1e4b-4141-a321-fd902abfeb28 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:53.575747 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-786a7957-1e4b-4141-a321-fd902abfeb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:53.588493 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:53.644126 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1368/2736] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:35:53.655834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178db52-0255-459b-8c4e-3816a4c8f771 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:53.657729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2178db52-0255-459b-8c4e-3816a4c8f771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:53.668953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:53.725681 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178db52-0255-459b-8c4e-3816a4c8f771 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:53.729570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb6d03-1d6d-4aac-aca7-3ec3675339de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:53.730185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb6d03-1d6d-4aac-aca7-3ec3675339de service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:53.730465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb6d03-1d6d-4aac-aca7-3ec3675339de service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:53.730739 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb6d03-1d6d-4aac-aca7-3ec3675339de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:53.740133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb6d03-1d6d-4aac-aca7-3ec3675339de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:53.741562 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb6d03-1d6d-4aac-aca7-3ec3675339de service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:53.751882 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1369/2737] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:35:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:35:54.283839 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a7957-1e4b-4141-a321-fd902abfeb28 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:54.285456 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-786a7957-1e4b-4141-a321-fd902abfeb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:54.297572 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a7957-1e4b-4141-a321-fd902abfeb28 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:54.302899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77cdf8-c27a-4377-bdcf-78fbd89cab8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:54.303548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77cdf8-c27a-4377-bdcf-78fbd89cab8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:54.303833 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77cdf8-c27a-4377-bdcf-78fbd89cab8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:54.304108 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77cdf8-c27a-4377-bdcf-78fbd89cab8b admin admin] PATH_INFO: `/v3/projects/a3812d2edd4e4b5989a1d2834d74c7e0` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:54.305232 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77cdf8-c27a-4377-bdcf-78fbd89cab8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3812d2edd4e4b5989a1d2834d74c7e0)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:54.307137 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77cdf8-c27a-4377-bdcf-78fbd89cab8b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:54.363002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fa77cdf8-c27a-4377-bdcf-78fbd89cab8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3812d2edd4e4b5989a1d2834d74c7e0', 'request_id': 'req-fa77cdf8-c27a-4377-bdcf-78fbd89cab8b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:54.370703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fa77cdf8-c27a-4377-bdcf-78fbd89cab8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3812d2edd4e4b5989a1d2834d74c7e0', 'request_id': 'req-fa77cdf8-c27a-4377-bdcf-78fbd89cab8b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:54.377419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fa77cdf8-c27a-4377-bdcf-78fbd89cab8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3812d2edd4e4b5989a1d2834d74c7e0', 'request_id': 'req-fa77cdf8-c27a-4377-bdcf-78fbd89cab8b'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:54.388271 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fa77cdf8-c27a-4377-bdcf-78fbd89cab8b admin admin] The token cache is being invalidate because project a3812d2edd4e4b5989a1d2834d74c7e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:35:54.388564 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fa77cdf8-c27a-4377-bdcf-78fbd89cab8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:54.389731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fa77cdf8-c27a-4377-bdcf-78fbd89cab8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:35:54.394455 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1369/2738] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:35:54 2022] DELETE /identity/v3/projects/a3812d2edd4e4b5989a1d2834d74c7e0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:54.396136 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150137e-bd17-4ca4-9ac2-869e328ab797 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:54.396537 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150137e-bd17-4ca4-9ac2-869e328ab797 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:54.396916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150137e-bd17-4ca4-9ac2-869e328ab797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:54.417074 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c150137e-bd17-4ca4-9ac2-869e328ab797 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:54.418807 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c150137e-bd17-4ca4-9ac2-869e328ab797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:54.486063 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1370/2739] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:35:54.493695 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf5dd-ebc3-4de2-ad71-36702fc1eed0 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:54.494006 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf5dd-ebc3-4de2-ad71-36702fc1eed0 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:54.494285 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf5dd-ebc3-4de2-ad71-36702fc1eed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:54.514389 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-47aaf5dd-ebc3-4de2-ad71-36702fc1eed0 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:54.515667 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-47aaf5dd-ebc3-4de2-ad71-36702fc1eed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:54.523808 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1370/2740] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:35:54.527811 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150137e-bd17-4ca4-9ac2-869e328ab797 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:54.538996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150137e-bd17-4ca4-9ac2-869e328ab797 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:54.543924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c47910-97d8-4475-af4d-1c217c25a54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:54.544540 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c47910-97d8-4475-af4d-1c217c25a54c admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:54.544825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c47910-97d8-4475-af4d-1c217c25a54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:54.545145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c47910-97d8-4475-af4d-1c217c25a54c admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:54.545802 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c47910-97d8-4475-af4d-1c217c25a54c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:35:54.546139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c47910-97d8-4475-af4d-1c217c25a54c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:54.547803 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c47910-97d8-4475-af4d-1c217c25a54c admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:54.564263 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1371/2741] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:35:54.565384 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaf5dd-ebc3-4de2-ad71-36702fc1eed0 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:54.575549 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaf5dd-ebc3-4de2-ad71-36702fc1eed0 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:54.580712 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d7e6d-3e26-4761-98d0-6417c0300aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:54.581325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d7e6d-3e26-4761-98d0-6417c0300aed admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:54.581653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d7e6d-3e26-4761-98d0-6417c0300aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:54.581873 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d7e6d-3e26-4761-98d0-6417c0300aed admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:54.582515 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d7e6d-3e26-4761-98d0-6417c0300aed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:54.584489 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d7e6d-3e26-4761-98d0-6417c0300aed admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:54.617225 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1371/2742] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:35:54 2022] POST /identity/v3/projects => generated 399 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:35:54.618577 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c47910-97d8-4475-af4d-1c217c25a54c admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:54.642402 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c47910-97d8-4475-af4d-1c217c25a54c admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:54.704383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d515ae-1693-4447-afd2-c89c103cff90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:54.705042 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d515ae-1693-4447-afd2-c89c103cff90 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:54.705295 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d515ae-1693-4447-afd2-c89c103cff90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:54.705573 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d515ae-1693-4447-afd2-c89c103cff90 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:54.706203 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d515ae-1693-4447-afd2-c89c103cff90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:54.708147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d515ae-1693-4447-afd2-c89c103cff90 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:54.731359 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1372/2743] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:35:54 2022] POST /identity/v3/users => generated 300 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:35:54.732611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d7e6d-3e26-4761-98d0-6417c0300aed admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:54.743071 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d7e6d-3e26-4761-98d0-6417c0300aed admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:54.747691 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22781ece-6dfc-4731-80f2-d62e5dfe6cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:54.748814 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22781ece-6dfc-4731-80f2-d62e5dfe6cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:54.749089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22781ece-6dfc-4731-80f2-d62e5dfe6cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:54.749356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22781ece-6dfc-4731-80f2-d62e5dfe6cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:54.750020 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22781ece-6dfc-4731-80f2-d62e5dfe6cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:54.751640 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22781ece-6dfc-4731-80f2-d62e5dfe6cc8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:54.762627 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1372/2744] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:35:54.767220 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d515ae-1693-4447-afd2-c89c103cff90 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:54.777251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d515ae-1693-4447-afd2-c89c103cff90 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:54.782189 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9708d2b-a1c9-4ee2-993e-02ad46d7232d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:54.782780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9708d2b-a1c9-4ee2-993e-02ad46d7232d admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:54.783061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9708d2b-a1c9-4ee2-993e-02ad46d7232d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:54.783332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9708d2b-a1c9-4ee2-993e-02ad46d7232d admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:54.784070 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9708d2b-a1c9-4ee2-993e-02ad46d7232d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:54.785678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9708d2b-a1c9-4ee2-993e-02ad46d7232d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:54.795473 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1373/2745] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:35:54.799562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22781ece-6dfc-4731-80f2-d62e5dfe6cc8 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:54.809796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22781ece-6dfc-4731-80f2-d62e5dfe6cc8 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:54.814909 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85895dca-367a-4a36-b67c-11a4c8d9d579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:54.815585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85895dca-367a-4a36-b67c-11a4c8d9d579 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:54.815877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85895dca-367a-4a36-b67c-11a4c8d9d579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:54.816154 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85895dca-367a-4a36-b67c-11a4c8d9d579 admin admin] PATH_INFO: `/v3/projects/e23621d60bbd4fd1aa76116be19ad226/users/0003888fbb63406ba0efb06124615a09/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:54.831082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85895dca-367a-4a36-b67c-11a4c8d9d579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e23621d60bbd4fd1aa76116be19ad226, user_id=0003888fbb63406ba0efb06124615a09, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:54.833754 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85895dca-367a-4a36-b67c-11a4c8d9d579 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:54.846643 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1373/2746] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:35:54 2022] PUT /identity/v3/projects/e23621d60bbd4fd1aa76116be19ad226/users/0003888fbb63406ba0efb06124615a09/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:35:54.858148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee589a7-6c4c-4af6-8588-b7ee9d8117bf None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:54.858527 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee589a7-6c4c-4af6-8588-b7ee9d8117bf None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:54.858860 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee589a7-6c4c-4af6-8588-b7ee9d8117bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:54.887733 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-4ee589a7-6c4c-4af6-8588-b7ee9d8117bf None None] MFA Rules not processed for user `0003888fbb63406ba0efb06124615a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:35:54.889559 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4ee589a7-6c4c-4af6-8588-b7ee9d8117bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:54.894637 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:54.959834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1374/2747] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:35:54 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:35:54.971603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85895dca-367a-4a36-b67c-11a4c8d9d579 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:35:54.973638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-85895dca-367a-4a36-b67c-11a4c8d9d579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:35:55.022780 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:35:55.042454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85895dca-367a-4a36-b67c-11a4c8d9d579 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:35:55.046836 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18764fc0-4046-41a8-8166-70be6b30ef59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:35:55.047441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18764fc0-4046-41a8-8166-70be6b30ef59 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:35:55.047757 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18764fc0-4046-41a8-8166-70be6b30ef59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:35:55.048031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18764fc0-4046-41a8-8166-70be6b30ef59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:35:55.061376 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18764fc0-4046-41a8-8166-70be6b30ef59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:35:55.063834 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18764fc0-4046-41a8-8166-70be6b30ef59 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:35:55.073373 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1374/2748] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:35:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:36:01.628889 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee589a7-6c4c-4af6-8588-b7ee9d8117bf None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:36:01.630323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-4ee589a7-6c4c-4af6-8588-b7ee9d8117bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:36:01.685716 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee589a7-6c4c-4af6-8588-b7ee9d8117bf None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:36:01.689460 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ab665-2471-43b1-9213-a01fad46f39d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:36:01.690104 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ab665-2471-43b1-9213-a01fad46f39d service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:36:01.690413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ab665-2471-43b1-9213-a01fad46f39d service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:36:01.690688 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ab665-2471-43b1-9213-a01fad46f39d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:36:01.692465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-965ab665-2471-43b1-9213-a01fad46f39d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:36:01.752483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ab665-2471-43b1-9213-a01fad46f39d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:36:01.753842 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ab665-2471-43b1-9213-a01fad46f39d service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:36:01.763444 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1375/2749] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:36:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:36:04.174472 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18764fc0-4046-41a8-8166-70be6b30ef59 service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:36:04.185242 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18764fc0-4046-41a8-8166-70be6b30ef59 service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:36:04.191199 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640998f7-47fb-44a6-84d0-3f5bb0c2e627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:36:04.192177 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640998f7-47fb-44a6-84d0-3f5bb0c2e627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:36:04.192715 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640998f7-47fb-44a6-84d0-3f5bb0c2e627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:36:04.193210 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640998f7-47fb-44a6-84d0-3f5bb0c2e627 admin admin] PATH_INFO: `/v3/users/0003888fbb63406ba0efb06124615a09` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:36:04.194585 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640998f7-47fb-44a6-84d0-3f5bb0c2e627 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0003888fbb63406ba0efb06124615a09)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:36:04.197336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640998f7-47fb-44a6-84d0-3f5bb0c2e627 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:36:04.268188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-640998f7-47fb-44a6-84d0-3f5bb0c2e627 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0003888fbb63406ba0efb06124615a09'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:04.276157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-640998f7-47fb-44a6-84d0-3f5bb0c2e627 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0003888fbb63406ba0efb06124615a09'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:04.276845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-640998f7-47fb-44a6-84d0-3f5bb0c2e627 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0003888fbb63406ba0efb06124615a09'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:04.277576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-640998f7-47fb-44a6-84d0-3f5bb0c2e627 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0003888fbb63406ba0efb06124615a09'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:04.286419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-640998f7-47fb-44a6-84d0-3f5bb0c2e627 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0003888fbb63406ba0efb06124615a09'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:04.304613 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1375/2750] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:36:04 2022] DELETE /identity/v3/users/0003888fbb63406ba0efb06124615a09 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:36:04.309209 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0e6b8-f0db-4810-9ea9-8b2f565f535c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:36:04.309697 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0e6b8-f0db-4810-9ea9-8b2f565f535c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:36:04.310064 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0e6b8-f0db-4810-9ea9-8b2f565f535c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:36:04.332346 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-efd0e6b8-f0db-4810-9ea9-8b2f565f535c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:36:04.334305 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-efd0e6b8-f0db-4810-9ea9-8b2f565f535c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:36:04.393815 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=100 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:36:04.411993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1376/2751] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:36:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:36:04.420857 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640998f7-47fb-44a6-84d0-3f5bb0c2e627 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:36:04.422449 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-640998f7-47fb-44a6-84d0-3f5bb0c2e627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:36:04.486411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640998f7-47fb-44a6-84d0-3f5bb0c2e627 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:36:04.490335 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a5c70-ce25-4668-8da8-20eaf32acff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:36:04.490962 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a5c70-ce25-4668-8da8-20eaf32acff0 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:36:04.491240 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a5c70-ce25-4668-8da8-20eaf32acff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:36:04.491548 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a5c70-ce25-4668-8da8-20eaf32acff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:36:04.501252 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a5c70-ce25-4668-8da8-20eaf32acff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:36:04.502604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a5c70-ce25-4668-8da8-20eaf32acff0 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:36:04.512110 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1376/2752] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:36:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:36:04.822311 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0e6b8-f0db-4810-9ea9-8b2f565f535c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:36:04.826015 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a5c70-ce25-4668-8da8-20eaf32acff0 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:36:04.827781 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-efd0e6b8-f0db-4810-9ea9-8b2f565f535c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:36:04.832308 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9c7a5c70-ce25-4668-8da8-20eaf32acff0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:36:04.850472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0e6b8-f0db-4810-9ea9-8b2f565f535c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:36:04.873735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025c274-f4d5-48c4-8bd7-70ad62f77a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:36:04.874988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025c274-f4d5-48c4-8bd7-70ad62f77a62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:36:04.876661 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025c274-f4d5-48c4-8bd7-70ad62f77a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:36:04.876960 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025c274-f4d5-48c4-8bd7-70ad62f77a62 admin admin] PATH_INFO: `/v3/projects/e23621d60bbd4fd1aa76116be19ad226` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:36:04.882604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025c274-f4d5-48c4-8bd7-70ad62f77a62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e23621d60bbd4fd1aa76116be19ad226)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:36:04.886462 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025c274-f4d5-48c4-8bd7-70ad62f77a62 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:36:04.931947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:36:04.949430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5025c274-f4d5-48c4-8bd7-70ad62f77a62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e23621d60bbd4fd1aa76116be19ad226', 'request_id': 'req-5025c274-f4d5-48c4-8bd7-70ad62f77a62'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:04.952115 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a5c70-ce25-4668-8da8-20eaf32acff0 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:36:04.962249 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5025c274-f4d5-48c4-8bd7-70ad62f77a62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e23621d60bbd4fd1aa76116be19ad226', 'request_id': 'req-5025c274-f4d5-48c4-8bd7-70ad62f77a62'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:04.970016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5025c274-f4d5-48c4-8bd7-70ad62f77a62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e23621d60bbd4fd1aa76116be19ad226', 'request_id': 'req-5025c274-f4d5-48c4-8bd7-70ad62f77a62'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:04.982947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5025c274-f4d5-48c4-8bd7-70ad62f77a62 admin admin] The token cache is being invalidate because project e23621d60bbd4fd1aa76116be19ad226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:36:04.983241 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5025c274-f4d5-48c4-8bd7-70ad62f77a62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:04.984049 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5025c274-f4d5-48c4-8bd7-70ad62f77a62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:04.985943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1377/2753] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:36:04 2022] DELETE /identity/v3/projects/e23621d60bbd4fd1aa76116be19ad226 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:36:05.052620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054e620-3d10-457c-bd70-8cb8c43c829a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:36:05.053359 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054e620-3d10-457c-bd70-8cb8c43c829a service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:36:05.053719 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054e620-3d10-457c-bd70-8cb8c43c829a service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:36:05.054070 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054e620-3d10-457c-bd70-8cb8c43c829a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:36:05.056288 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7054e620-3d10-457c-bd70-8cb8c43c829a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:36:05.070824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054e620-3d10-457c-bd70-8cb8c43c829a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:36:05.072659 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054e620-3d10-457c-bd70-8cb8c43c829a service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:36:05.085058 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1377/2754] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:36:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:36:08.267533 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025c274-f4d5-48c4-8bd7-70ad62f77a62 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:36:08.269087 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5025c274-f4d5-48c4-8bd7-70ad62f77a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:36:08.334642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025c274-f4d5-48c4-8bd7-70ad62f77a62 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:36:08.339292 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce3eeb-c507-4a97-8371-11a7c0d2f17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:36:08.339959 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce3eeb-c507-4a97-8371-11a7c0d2f17e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:36:08.340242 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce3eeb-c507-4a97-8371-11a7c0d2f17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:36:08.340514 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce3eeb-c507-4a97-8371-11a7c0d2f17e admin admin] PATH_INFO: `/v3/users/dff0daad12c143fabd16d40a1939b87e` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:36:08.341618 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce3eeb-c507-4a97-8371-11a7c0d2f17e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dff0daad12c143fabd16d40a1939b87e)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:36:08.343617 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce3eeb-c507-4a97-8371-11a7c0d2f17e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:36:08.389365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d7ce3eeb-c507-4a97-8371-11a7c0d2f17e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dff0daad12c143fabd16d40a1939b87e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:08.395951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d7ce3eeb-c507-4a97-8371-11a7c0d2f17e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dff0daad12c143fabd16d40a1939b87e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:08.396603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d7ce3eeb-c507-4a97-8371-11a7c0d2f17e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dff0daad12c143fabd16d40a1939b87e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:08.397218 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d7ce3eeb-c507-4a97-8371-11a7c0d2f17e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dff0daad12c143fabd16d40a1939b87e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:08.403968 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d7ce3eeb-c507-4a97-8371-11a7c0d2f17e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dff0daad12c143fabd16d40a1939b87e'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:08.417528 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:36:08.422540 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1378/2755] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:36:08 2022] DELETE /identity/v3/users/dff0daad12c143fabd16d40a1939b87e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:36:08.590442 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054e620-3d10-457c-bd70-8cb8c43c829a service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:36:08.606437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054e620-3d10-457c-bd70-8cb8c43c829a service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:36:08.663769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed954d46-ee81-4172-b1f4-c1a27e2ac871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:36:08.666447 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed954d46-ee81-4172-b1f4-c1a27e2ac871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:36:08.666690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed954d46-ee81-4172-b1f4-c1a27e2ac871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:36:08.666962 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed954d46-ee81-4172-b1f4-c1a27e2ac871 admin admin] PATH_INFO: `/v3/projects/c80feb95ecbf413ca0bc52a519bf43b7` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:36:08.669089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed954d46-ee81-4172-b1f4-c1a27e2ac871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c80feb95ecbf413ca0bc52a519bf43b7)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:36:08.671540 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed954d46-ee81-4172-b1f4-c1a27e2ac871 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:36:08.718286 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ed954d46-ee81-4172-b1f4-c1a27e2ac871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c80feb95ecbf413ca0bc52a519bf43b7', 'request_id': 'req-ed954d46-ee81-4172-b1f4-c1a27e2ac871'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:08.725305 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ed954d46-ee81-4172-b1f4-c1a27e2ac871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c80feb95ecbf413ca0bc52a519bf43b7', 'request_id': 'req-ed954d46-ee81-4172-b1f4-c1a27e2ac871'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:08.731237 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ed954d46-ee81-4172-b1f4-c1a27e2ac871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c80feb95ecbf413ca0bc52a519bf43b7', 'request_id': 'req-ed954d46-ee81-4172-b1f4-c1a27e2ac871'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:08.741780 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ed954d46-ee81-4172-b1f4-c1a27e2ac871 admin admin] The token cache is being invalidate because project c80feb95ecbf413ca0bc52a519bf43b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:36:08.742064 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ed954d46-ee81-4172-b1f4-c1a27e2ac871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:08.742679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ed954d46-ee81-4172-b1f4-c1a27e2ac871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:08.744212 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1378/2756] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:36:08 2022] DELETE /identity/v3/projects/c80feb95ecbf413ca0bc52a519bf43b7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:36:08.748881 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce3eeb-c507-4a97-8371-11a7c0d2f17e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:36:08.750289 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d7ce3eeb-c507-4a97-8371-11a7c0d2f17e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:36:08.763166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce3eeb-c507-4a97-8371-11a7c0d2f17e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:36:08.820208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a6544-3ebe-4be4-8c9b-4612ca6df7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:36:08.820867 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a6544-3ebe-4be4-8c9b-4612ca6df7f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:36:08.821152 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a6544-3ebe-4be4-8c9b-4612ca6df7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:36:08.821482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a6544-3ebe-4be4-8c9b-4612ca6df7f7 admin admin] PATH_INFO: `/v3/users/a237896782e9404495e06813abb74a7b` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:36:08.835694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a6544-3ebe-4be4-8c9b-4612ca6df7f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a237896782e9404495e06813abb74a7b)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:36:08.837634 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a6544-3ebe-4be4-8c9b-4612ca6df7f7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:36:08.882534 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-451a6544-3ebe-4be4-8c9b-4612ca6df7f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a237896782e9404495e06813abb74a7b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:08.890408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-451a6544-3ebe-4be4-8c9b-4612ca6df7f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a237896782e9404495e06813abb74a7b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:08.891107 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-451a6544-3ebe-4be4-8c9b-4612ca6df7f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a237896782e9404495e06813abb74a7b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:08.891753 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-451a6544-3ebe-4be4-8c9b-4612ca6df7f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a237896782e9404495e06813abb74a7b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:08.899775 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-451a6544-3ebe-4be4-8c9b-4612ca6df7f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a237896782e9404495e06813abb74a7b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:08.917776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1379/2757] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:36:08 2022] DELETE /identity/v3/users/a237896782e9404495e06813abb74a7b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:36:09.083381 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed954d46-ee81-4172-b1f4-c1a27e2ac871 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:36:09.084911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ed954d46-ee81-4172-b1f4-c1a27e2ac871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:36:09.098703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed954d46-ee81-4172-b1f4-c1a27e2ac871 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:36:09.112690 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:36:09.159096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:36:09.159740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:36:09.160018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:36:09.160289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1 admin admin] PATH_INFO: `/v3/projects/7d7cefa469664b099a6f0a24e705bdad` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:36:09.171446 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d7cefa469664b099a6f0a24e705bdad)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:36:09.173315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:36:09.218938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d7cefa469664b099a6f0a24e705bdad', 'request_id': 'req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:09.227717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d7cefa469664b099a6f0a24e705bdad', 'request_id': 'req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:09.234909 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d7cefa469664b099a6f0a24e705bdad', 'request_id': 'req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:09.254269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1 admin admin] The token cache is being invalidate because project 7d7cefa469664b099a6f0a24e705bdad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:36:09.254564 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:09.255223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:36:09.256796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1379/2758] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:36:09 2022] DELETE /identity/v3/projects/7d7cefa469664b099a6f0a24e705bdad => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:36:18.913958 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=58,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:36:19.247236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:36:50.937346 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8d5f9-9b7f-4a9f-a425-9e8cdeb5c68b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:36:50.937874 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8d5f9-9b7f-4a9f-a425-9e8cdeb5c68b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:36:50.937874 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8d5f9-9b7f-4a9f-a425-9e8cdeb5c68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:36:50.958205 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-63e8d5f9-9b7f-4a9f-a425-9e8cdeb5c68b None None] MFA Rules not processed for user `404c5e2b85454e1e91b90fa45796511e`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:36:50.959434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-63e8d5f9-9b7f-4a9f-a425-9e8cdeb5c68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:36:51.011840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1380/2759] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:36:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:37:01.005246 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=25 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:38:13.961473 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:38:13.962965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:38:14.019038 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73547ca2-3b25-4b1b-b64d-c2b5ba147bd1 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:38:14.022651 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cefa78-a532-4250-b4a7-9468f3fb5182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:38:14.023247 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cefa78-a532-4250-b4a7-9468f3fb5182 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:38:14.023554 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cefa78-a532-4250-b4a7-9468f3fb5182 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:38:14.023830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cefa78-a532-4250-b4a7-9468f3fb5182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:38:14.025528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-81cefa78-a532-4250-b4a7-9468f3fb5182 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:38:14.035636 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cefa78-a532-4250-b4a7-9468f3fb5182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:38:14.037061 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cefa78-a532-4250-b4a7-9468f3fb5182 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:38:14.044957 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1380/2760] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:38:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:38:20.739338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8d5f9-9b7f-4a9f-a425-9e8cdeb5c68b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:38:20.741225 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-63e8d5f9-9b7f-4a9f-a425-9e8cdeb5c68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:38:20.815712 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8d5f9-9b7f-4a9f-a425-9e8cdeb5c68b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:38:20.821472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2162afa-798c-4e9f-a6ee-abb9240acf22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:38:20.822083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2162afa-798c-4e9f-a6ee-abb9240acf22 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:38:20.822367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2162afa-798c-4e9f-a6ee-abb9240acf22 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:38:20.822642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2162afa-798c-4e9f-a6ee-abb9240acf22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:38:20.824375 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d2162afa-798c-4e9f-a6ee-abb9240acf22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:38:20.835391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2162afa-798c-4e9f-a6ee-abb9240acf22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:38:20.837025 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2162afa-798c-4e9f-a6ee-abb9240acf22 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:38:20.846502 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1381/2761] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:38:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:38:22.277924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cefa78-a532-4250-b4a7-9468f3fb5182 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:38:22.279322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-81cefa78-a532-4250-b4a7-9468f3fb5182 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:38:22.335144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cefa78-a532-4250-b4a7-9468f3fb5182 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:38:22.338677 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6b887-27fa-46a3-b0de-485b38cb79d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:38:22.339269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6b887-27fa-46a3-b0de-485b38cb79d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:38:22.339588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6b887-27fa-46a3-b0de-485b38cb79d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:38:22.339866 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6b887-27fa-46a3-b0de-485b38cb79d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:38:22.341539 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c1e6b887-27fa-46a3-b0de-485b38cb79d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:38:22.355113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6b887-27fa-46a3-b0de-485b38cb79d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:38:22.356597 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6b887-27fa-46a3-b0de-485b38cb79d2 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:38:22.367837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1381/2762] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:38:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:38:30.190656 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2162afa-798c-4e9f-a6ee-abb9240acf22 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:38:30.199943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2162afa-798c-4e9f-a6ee-abb9240acf22 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:38:30.203517 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5cff5-b9fd-4512-beb4-6730c6cd2b28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:38:30.204111 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5cff5-b9fd-4512-beb4-6730c6cd2b28 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:38:30.204385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5cff5-b9fd-4512-beb4-6730c6cd2b28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:38:30.204653 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5cff5-b9fd-4512-beb4-6730c6cd2b28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:38:30.206395 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-feb5cff5-b9fd-4512-beb4-6730c6cd2b28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:38:30.219166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5cff5-b9fd-4512-beb4-6730c6cd2b28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:38:30.220589 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5cff5-b9fd-4512-beb4-6730c6cd2b28 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:38:30.231367 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1382/2763] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:38:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:38:32.363075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=52 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:38:37.516655 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b024a-e8ff-4c11-8401-9e42a2786d42 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:38:37.517259 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b024a-e8ff-4c11-8401-9e42a2786d42 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:38:37.517259 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b024a-e8ff-4c11-8401-9e42a2786d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:38:37.536626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-c11b024a-e8ff-4c11-8401-9e42a2786d42 None None] MFA Rules not processed for user `404c5e2b85454e1e91b90fa45796511e`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:38:37.537822 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c11b024a-e8ff-4c11-8401-9e42a2786d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:38:37.544588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1382/2764] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:38:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:38:37.841615 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5cff5-b9fd-4512-beb4-6730c6cd2b28 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:38:37.851591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5cff5-b9fd-4512-beb4-6730c6cd2b28 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:38:37.855148 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca55e0-4355-49b0-b07f-95516359be8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:38:37.855772 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca55e0-4355-49b0-b07f-95516359be8d service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:38:37.856057 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca55e0-4355-49b0-b07f-95516359be8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:38:37.856330 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca55e0-4355-49b0-b07f-95516359be8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:38:37.858049 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-02ca55e0-4355-49b0-b07f-95516359be8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:38:37.867293 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca55e0-4355-49b0-b07f-95516359be8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:38:37.868752 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca55e0-4355-49b0-b07f-95516359be8d service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:38:37.876672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1383/2765] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:38:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:38:38.185307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b024a-e8ff-4c11-8401-9e42a2786d42 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:38:38.187241 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-c11b024a-e8ff-4c11-8401-9e42a2786d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:38:38.263892 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b024a-e8ff-4c11-8401-9e42a2786d42 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:38:38.269460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e8e58-e2db-4c54-affb-ea63a372ec7c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:38:38.270063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e8e58-e2db-4c54-affb-ea63a372ec7c service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:38:38.270348 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e8e58-e2db-4c54-affb-ea63a372ec7c service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:38:38.270622 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e8e58-e2db-4c54-affb-ea63a372ec7c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:38:38.272378 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-9c5e8e58-e2db-4c54-affb-ea63a372ec7c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:38:38.283686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e8e58-e2db-4c54-affb-ea63a372ec7c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:38:38.285509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e8e58-e2db-4c54-affb-ea63a372ec7c service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:38:38.295255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1383/2766] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:38:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:38:46.413134 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca55e0-4355-49b0-b07f-95516359be8d service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:38:46.421904 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=47 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:38:46.429481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca55e0-4355-49b0-b07f-95516359be8d service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:38:46.434962 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b115b9-8d45-4848-a8eb-8c3ef031a940 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:38:46.435688 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b115b9-8d45-4848-a8eb-8c3ef031a940 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:38:46.436031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b115b9-8d45-4848-a8eb-8c3ef031a940 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:38:46.436401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b115b9-8d45-4848-a8eb-8c3ef031a940 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:38:46.438183 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-06b115b9-8d45-4848-a8eb-8c3ef031a940 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:38:46.450197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b115b9-8d45-4848-a8eb-8c3ef031a940 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:38:46.451880 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b115b9-8d45-4848-a8eb-8c3ef031a940 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:38:46.462482 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1384/2767] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:38:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:38:48.291072 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=35 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:38:56.457764 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:24.978954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e8e58-e2db-4c54-affb-ea63a372ec7c service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:24.998167 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e8e58-e2db-4c54-affb-ea63a372ec7c service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:25.001815 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6760d4-e28e-441f-9a64-f5b490b46ba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:25.002421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6760d4-e28e-441f-9a64-f5b490b46ba5 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:25.002710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6760d4-e28e-441f-9a64-f5b490b46ba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:25.002999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6760d4-e28e-441f-9a64-f5b490b46ba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:25.005027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-aa6760d4-e28e-441f-9a64-f5b490b46ba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:25.103193 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6760d4-e28e-441f-9a64-f5b490b46ba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:25.104833 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6760d4-e28e-441f-9a64-f5b490b46ba5 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:25.116151 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1384/2768] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:40:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:40:28.082577 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b115b9-8d45-4848-a8eb-8c3ef031a940 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:28.085061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-06b115b9-8d45-4848-a8eb-8c3ef031a940 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:28.149872 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6760d4-e28e-441f-9a64-f5b490b46ba5 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:28.151787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-aa6760d4-e28e-441f-9a64-f5b490b46ba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:28.175826 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6760d4-e28e-441f-9a64-f5b490b46ba5 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:28.180683 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e5d82-b6ba-4a37-ad42-92260a7c5ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:28.181439 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e5d82-b6ba-4a37-ad42-92260a7c5ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:28.181732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e5d82-b6ba-4a37-ad42-92260a7c5ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:28.182065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e5d82-b6ba-4a37-ad42-92260a7c5ee3 admin admin] PATH_INFO: `/v3/users/404c5e2b85454e1e91b90fa45796511e` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:28.183194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e5d82-b6ba-4a37-ad42-92260a7c5ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=404c5e2b85454e1e91b90fa45796511e)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:28.185189 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e5d82-b6ba-4a37-ad42-92260a7c5ee3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:28.194089 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b115b9-8d45-4848-a8eb-8c3ef031a940 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:28.200095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b074d6-303a-4dc1-84fc-9da38ad92b0b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:28.200705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b074d6-303a-4dc1-84fc-9da38ad92b0b service placement] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:28.200994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b074d6-303a-4dc1-84fc-9da38ad92b0b service placement] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:28.201269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b074d6-303a-4dc1-84fc-9da38ad92b0b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:28.203019 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a0b074d6-303a-4dc1-84fc-9da38ad92b0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:28.215780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b074d6-303a-4dc1-84fc-9da38ad92b0b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:28.217377 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b074d6-303a-4dc1-84fc-9da38ad92b0b service placement] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:28.228426 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1385/2769] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:40:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:40:28.237716 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e13e5d82-b6ba-4a37-ad42-92260a7c5ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '404c5e2b85454e1e91b90fa45796511e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:28.245592 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e13e5d82-b6ba-4a37-ad42-92260a7c5ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '404c5e2b85454e1e91b90fa45796511e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:28.247031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e13e5d82-b6ba-4a37-ad42-92260a7c5ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '404c5e2b85454e1e91b90fa45796511e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:28.247553 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e13e5d82-b6ba-4a37-ad42-92260a7c5ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '404c5e2b85454e1e91b90fa45796511e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:28.257004 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-e13e5d82-b6ba-4a37-ad42-92260a7c5ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '404c5e2b85454e1e91b90fa45796511e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:28.275307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1385/2770] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:40:28 2022] DELETE /identity/v3/users/404c5e2b85454e1e91b90fa45796511e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:28.441164 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b074d6-303a-4dc1-84fc-9da38ad92b0b service placement] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:28.451565 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b074d6-303a-4dc1-84fc-9da38ad92b0b service placement] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:28.507417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f51703-dc59-48b2-a95b-429737d278b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:28.508066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f51703-dc59-48b2-a95b-429737d278b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:28.508344 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f51703-dc59-48b2-a95b-429737d278b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:28.508616 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f51703-dc59-48b2-a95b-429737d278b5 admin admin] PATH_INFO: `/v3/projects/bee9f30f6e1d4d1ab2c4fe269d78d768` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:28.509725 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f51703-dc59-48b2-a95b-429737d278b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bee9f30f6e1d4d1ab2c4fe269d78d768)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:28.511630 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f51703-dc59-48b2-a95b-429737d278b5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:28.556160 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-25f51703-dc59-48b2-a95b-429737d278b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bee9f30f6e1d4d1ab2c4fe269d78d768', 'request_id': 'req-25f51703-dc59-48b2-a95b-429737d278b5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:28.563077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-25f51703-dc59-48b2-a95b-429737d278b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bee9f30f6e1d4d1ab2c4fe269d78d768', 'request_id': 'req-25f51703-dc59-48b2-a95b-429737d278b5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:28.569208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-25f51703-dc59-48b2-a95b-429737d278b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bee9f30f6e1d4d1ab2c4fe269d78d768', 'request_id': 'req-25f51703-dc59-48b2-a95b-429737d278b5'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:28.579133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-25f51703-dc59-48b2-a95b-429737d278b5 admin admin] The token cache is being invalidate because project bee9f30f6e1d4d1ab2c4fe269d78d768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:40:28.579416 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-25f51703-dc59-48b2-a95b-429737d278b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:28.580062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-25f51703-dc59-48b2-a95b-429737d278b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:28.581529 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1386/2771] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:40:28 2022] DELETE /identity/v3/projects/bee9f30f6e1d4d1ab2c4fe269d78d768 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:28.586569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f139111-d048-41b2-8c92-43167a656417 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:28.586857 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f139111-d048-41b2-8c92-43167a656417 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:28.587122 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f139111-d048-41b2-8c92-43167a656417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:28.607555 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-1f139111-d048-41b2-8c92-43167a656417 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:28.608748 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1f139111-d048-41b2-8c92-43167a656417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:28.628015 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:28.668670 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1386/2772] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:28.676545 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c4c10-bfc4-4213-9c2a-a200f6d39141 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:28.676838 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c4c10-bfc4-4213-9c2a-a200f6d39141 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:28.677121 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c4c10-bfc4-4213-9c2a-a200f6d39141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:28.696646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-ee3c4c10-bfc4-4213-9c2a-a200f6d39141 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:28.697789 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ee3c4c10-bfc4-4213-9c2a-a200f6d39141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:28.706168 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1387/2773] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:28.710289 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f139111-d048-41b2-8c92-43167a656417 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:28.720633 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f139111-d048-41b2-8c92-43167a656417 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:28.725127 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ac955-d9b0-429c-9cfe-7996b099075b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:28.725717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ac955-d9b0-429c-9cfe-7996b099075b admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:28.725996 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ac955-d9b0-429c-9cfe-7996b099075b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:28.726266 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ac955-d9b0-429c-9cfe-7996b099075b admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:28.726898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ac955-d9b0-429c-9cfe-7996b099075b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:40:28.727197 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ac955-d9b0-429c-9cfe-7996b099075b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:28.728805 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ac955-d9b0-429c-9cfe-7996b099075b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:28.739959 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1387/2774] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:40:28.744065 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c4c10-bfc4-4213-9c2a-a200f6d39141 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:28.748540 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:28.754926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c4c10-bfc4-4213-9c2a-a200f6d39141 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:28.760798 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643dd68-144d-462d-8711-b7c728b285eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:28.761397 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643dd68-144d-462d-8711-b7c728b285eb admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:28.762216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643dd68-144d-462d-8711-b7c728b285eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:28.762491 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643dd68-144d-462d-8711-b7c728b285eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:28.763117 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643dd68-144d-462d-8711-b7c728b285eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:28.765073 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643dd68-144d-462d-8711-b7c728b285eb admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:28.792538 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1388/2775] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:40:28 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:28.796848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ac955-d9b0-429c-9cfe-7996b099075b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:28.806845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ac955-d9b0-429c-9cfe-7996b099075b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:28.863558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b92a81-92cc-4132-9003-adb6006b65a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:28.864169 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b92a81-92cc-4132-9003-adb6006b65a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:28.864451 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b92a81-92cc-4132-9003-adb6006b65a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:28.864725 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b92a81-92cc-4132-9003-adb6006b65a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:28.865368 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b92a81-92cc-4132-9003-adb6006b65a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:28.867273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b92a81-92cc-4132-9003-adb6006b65a7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:28.889149 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1388/2776] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:40:28 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:28.893478 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643dd68-144d-462d-8711-b7c728b285eb admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:28.904619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643dd68-144d-462d-8711-b7c728b285eb admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:28.909119 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997772d-56ad-4ba8-927c-34f0ad5d9482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:28.909709 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997772d-56ad-4ba8-927c-34f0ad5d9482 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:28.909985 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997772d-56ad-4ba8-927c-34f0ad5d9482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:28.910257 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997772d-56ad-4ba8-927c-34f0ad5d9482 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:28.911152 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997772d-56ad-4ba8-927c-34f0ad5d9482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:28.912774 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997772d-56ad-4ba8-927c-34f0ad5d9482 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:28.922902 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1389/2777] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:28.926788 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b92a81-92cc-4132-9003-adb6006b65a7 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:28.937157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b92a81-92cc-4132-9003-adb6006b65a7 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:28.941674 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda7a24-c0ca-4e5d-929d-fc450f562e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:28.942257 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda7a24-c0ca-4e5d-929d-fc450f562e90 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:28.942554 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda7a24-c0ca-4e5d-929d-fc450f562e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:28.942825 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda7a24-c0ca-4e5d-929d-fc450f562e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:28.943468 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda7a24-c0ca-4e5d-929d-fc450f562e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:28.945032 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda7a24-c0ca-4e5d-929d-fc450f562e90 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:28.955602 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1389/2778] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:28.959420 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997772d-56ad-4ba8-927c-34f0ad5d9482 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:28.969036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997772d-56ad-4ba8-927c-34f0ad5d9482 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:28.973923 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4d33f-507f-4387-9cfa-bc5a21510b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:28.974569 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4d33f-507f-4387-9cfa-bc5a21510b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:28.974854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4d33f-507f-4387-9cfa-bc5a21510b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:28.975132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4d33f-507f-4387-9cfa-bc5a21510b32 admin admin] PATH_INFO: `/v3/projects/c11ff59d05cd492ea27ecd1006f9f44f/users/01bb2dd32a5d4882a86d3751bf34fe59/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:28.990066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4d33f-507f-4387-9cfa-bc5a21510b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c11ff59d05cd492ea27ecd1006f9f44f, user_id=01bb2dd32a5d4882a86d3751bf34fe59, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:28.992736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4d33f-507f-4387-9cfa-bc5a21510b32 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:29.006085 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1390/2779] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:40:28 2022] PUT /identity/v3/projects/c11ff59d05cd492ea27ecd1006f9f44f/users/01bb2dd32a5d4882a86d3751bf34fe59/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:29.011064 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69206d9b-ac31-4ad8-93f2-da10bd190d23 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:29.011349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69206d9b-ac31-4ad8-93f2-da10bd190d23 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:29.011642 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69206d9b-ac31-4ad8-93f2-da10bd190d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:29.030761 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-69206d9b-ac31-4ad8-93f2-da10bd190d23 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:29.032002 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-69206d9b-ac31-4ad8-93f2-da10bd190d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:29.089079 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1390/2780] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:29.099066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4d33f-507f-4387-9cfa-bc5a21510b32 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:29.100865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6fd4d33f-507f-4387-9cfa-bc5a21510b32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:29.157638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4d33f-507f-4387-9cfa-bc5a21510b32 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:29.161544 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b13a00-104e-4668-ae57-ff7b7f2a7f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:29.162175 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b13a00-104e-4668-ae57-ff7b7f2a7f66 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:29.162459 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b13a00-104e-4668-ae57-ff7b7f2a7f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:29.162735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b13a00-104e-4668-ae57-ff7b7f2a7f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:29.172235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b13a00-104e-4668-ae57-ff7b7f2a7f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:29.173658 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b13a00-104e-4668-ae57-ff7b7f2a7f66 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:29.182891 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1391/2781] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:40:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:40:29.796732 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc4fd8-a41a-4b23-9cf2-d2028591710d None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:29.797215 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc4fd8-a41a-4b23-9cf2-d2028591710d None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:29.797322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc4fd8-a41a-4b23-9cf2-d2028591710d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:29.826260 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-ecdc4fd8-a41a-4b23-9cf2-d2028591710d None None] MFA Rules not processed for user `01bb2dd32a5d4882a86d3751bf34fe59`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:29.827597 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ecdc4fd8-a41a-4b23-9cf2-d2028591710d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:29.848532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:29.895983 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1391/2782] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:29 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:40:29.900468 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b13a00-104e-4668-ae57-ff7b7f2a7f66 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:29.910502 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b13a00-104e-4668-ae57-ff7b7f2a7f66 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:29.915326 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579b2df-29cd-43ff-91a6-107a44c50aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:29.915953 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579b2df-29cd-43ff-91a6-107a44c50aba admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:29.916240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579b2df-29cd-43ff-91a6-107a44c50aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:29.916520 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579b2df-29cd-43ff-91a6-107a44c50aba admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:29.917152 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579b2df-29cd-43ff-91a6-107a44c50aba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:29.919639 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579b2df-29cd-43ff-91a6-107a44c50aba admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:29.947928 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1392/2783] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:40:29 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:29.951604 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc4fd8-a41a-4b23-9cf2-d2028591710d None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:29.963291 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc4fd8-a41a-4b23-9cf2-d2028591710d None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:30.020411 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb59ae-6571-4d3b-98cc-6e44213184f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:30.020953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb59ae-6571-4d3b-98cc-6e44213184f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:30.021236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb59ae-6571-4d3b-98cc-6e44213184f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:30.021510 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb59ae-6571-4d3b-98cc-6e44213184f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:30.022148 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb59ae-6571-4d3b-98cc-6e44213184f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:30.024099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb59ae-6571-4d3b-98cc-6e44213184f0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:30.049938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1392/2784] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:40:29 2022] POST /identity/v3/users => generated 308 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:30.051159 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579b2df-29cd-43ff-91a6-107a44c50aba admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:30.061642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579b2df-29cd-43ff-91a6-107a44c50aba admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:30.066323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc193dc-967d-442f-86d5-82c0462f264a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:30.066919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc193dc-967d-442f-86d5-82c0462f264a admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:30.067205 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc193dc-967d-442f-86d5-82c0462f264a admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:30.067483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc193dc-967d-442f-86d5-82c0462f264a admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:30.068161 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc193dc-967d-442f-86d5-82c0462f264a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:30.069746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc193dc-967d-442f-86d5-82c0462f264a admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:30.080149 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1393/2785] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:40:30.084391 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb59ae-6571-4d3b-98cc-6e44213184f0 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:30.094812 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb59ae-6571-4d3b-98cc-6e44213184f0 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:30.099727 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c4694-29ad-49e2-9518-6edd5109892a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:30.100574 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c4694-29ad-49e2-9518-6edd5109892a admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:30.100872 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c4694-29ad-49e2-9518-6edd5109892a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:30.101174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c4694-29ad-49e2-9518-6edd5109892a admin admin] PATH_INFO: `/v3/projects/6bd5aa0898834598ac30a381d4914cd9/users/49b36a128fb440d28df0b7e39420b041/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:30.117118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c4694-29ad-49e2-9518-6edd5109892a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bd5aa0898834598ac30a381d4914cd9, user_id=49b36a128fb440d28df0b7e39420b041, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:30.119795 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c4694-29ad-49e2-9518-6edd5109892a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:30.132059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1393/2786] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:40:30 2022] PUT /identity/v3/projects/6bd5aa0898834598ac30a381d4914cd9/users/49b36a128fb440d28df0b7e39420b041/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:40:30.135943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc193dc-967d-442f-86d5-82c0462f264a admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:30.146328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc193dc-967d-442f-86d5-82c0462f264a admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:30.203347 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f04f50-5203-4cc5-b0b8-2d3c33439bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:30.203977 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f04f50-5203-4cc5-b0b8-2d3c33439bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:30.204254 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f04f50-5203-4cc5-b0b8-2d3c33439bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:30.204526 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f04f50-5203-4cc5-b0b8-2d3c33439bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:30.205152 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f04f50-5203-4cc5-b0b8-2d3c33439bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:30.206723 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f04f50-5203-4cc5-b0b8-2d3c33439bb0 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:30.216047 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1394/2787] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:30.220276 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c4694-29ad-49e2-9518-6edd5109892a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:30.231041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c4694-29ad-49e2-9518-6edd5109892a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:30.235606 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b68f0-9fa1-473f-8013-71cabdb9013a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:30.236251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b68f0-9fa1-473f-8013-71cabdb9013a admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:30.236532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b68f0-9fa1-473f-8013-71cabdb9013a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:30.236807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b68f0-9fa1-473f-8013-71cabdb9013a admin admin] PATH_INFO: `/v3/projects/6bd5aa0898834598ac30a381d4914cd9/users/49b36a128fb440d28df0b7e39420b041/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:30.238762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b68f0-9fa1-473f-8013-71cabdb9013a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bd5aa0898834598ac30a381d4914cd9, user_id=49b36a128fb440d28df0b7e39420b041, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:30.241421 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b68f0-9fa1-473f-8013-71cabdb9013a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:30.252460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1394/2788] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:40:30 2022] PUT /identity/v3/projects/6bd5aa0898834598ac30a381d4914cd9/users/49b36a128fb440d28df0b7e39420b041/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:30.859419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27840da1-f688-4c4e-886c-769812523da7 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:30.859748 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27840da1-f688-4c4e-886c-769812523da7 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:30.860021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27840da1-f688-4c4e-886c-769812523da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:30.883802 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:30.896708 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-27840da1-f688-4c4e-886c-769812523da7 None None] MFA Rules not processed for user `49b36a128fb440d28df0b7e39420b041`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:30.899004 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-27840da1-f688-4c4e-886c-769812523da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:30.971432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1395/2789] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:30 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:31.971383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b68f0-9fa1-473f-8013-71cabdb9013a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:32.033009 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b68f0-9fa1-473f-8013-71cabdb9013a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:32.037743 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac408d5d-4d0e-47ae-981d-78f3f1ad8e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:32.038353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac408d5d-4d0e-47ae-981d-78f3f1ad8e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:32.038629 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac408d5d-4d0e-47ae-981d-78f3f1ad8e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:32.038897 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac408d5d-4d0e-47ae-981d-78f3f1ad8e0d admin admin] PATH_INFO: `/v3/users/01bb2dd32a5d4882a86d3751bf34fe59` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:32.040018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac408d5d-4d0e-47ae-981d-78f3f1ad8e0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=01bb2dd32a5d4882a86d3751bf34fe59)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:32.041901 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac408d5d-4d0e-47ae-981d-78f3f1ad8e0d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:32.056914 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:32.090405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ac408d5d-4d0e-47ae-981d-78f3f1ad8e0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01bb2dd32a5d4882a86d3751bf34fe59'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.097437 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ac408d5d-4d0e-47ae-981d-78f3f1ad8e0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01bb2dd32a5d4882a86d3751bf34fe59'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.098433 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ac408d5d-4d0e-47ae-981d-78f3f1ad8e0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01bb2dd32a5d4882a86d3751bf34fe59'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.099084 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ac408d5d-4d0e-47ae-981d-78f3f1ad8e0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01bb2dd32a5d4882a86d3751bf34fe59'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.106931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ac408d5d-4d0e-47ae-981d-78f3f1ad8e0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01bb2dd32a5d4882a86d3751bf34fe59'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.123875 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1395/2790] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:40:31 2022] DELETE /identity/v3/users/01bb2dd32a5d4882a86d3751bf34fe59 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:32.290417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27840da1-f688-4c4e-886c-769812523da7 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:32.305149 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27840da1-f688-4c4e-886c-769812523da7 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:32.363643 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c431c8-d83e-4b4c-b469-1f29d3aab74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:32.364276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c431c8-d83e-4b4c-b469-1f29d3aab74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:32.364556 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c431c8-d83e-4b4c-b469-1f29d3aab74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:32.364829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c431c8-d83e-4b4c-b469-1f29d3aab74b admin admin] PATH_INFO: `/v3/projects/c11ff59d05cd492ea27ecd1006f9f44f` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:32.365932 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c431c8-d83e-4b4c-b469-1f29d3aab74b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c11ff59d05cd492ea27ecd1006f9f44f)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:32.367837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c431c8-d83e-4b4c-b469-1f29d3aab74b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:32.413025 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-39c431c8-d83e-4b4c-b469-1f29d3aab74b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c11ff59d05cd492ea27ecd1006f9f44f', 'request_id': 'req-39c431c8-d83e-4b4c-b469-1f29d3aab74b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.421618 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-39c431c8-d83e-4b4c-b469-1f29d3aab74b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c11ff59d05cd492ea27ecd1006f9f44f', 'request_id': 'req-39c431c8-d83e-4b4c-b469-1f29d3aab74b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.428064 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-39c431c8-d83e-4b4c-b469-1f29d3aab74b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c11ff59d05cd492ea27ecd1006f9f44f', 'request_id': 'req-39c431c8-d83e-4b4c-b469-1f29d3aab74b'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.438679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-39c431c8-d83e-4b4c-b469-1f29d3aab74b admin admin] The token cache is being invalidate because project c11ff59d05cd492ea27ecd1006f9f44f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:40:32.438969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-39c431c8-d83e-4b4c-b469-1f29d3aab74b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.439658 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-39c431c8-d83e-4b4c-b469-1f29d3aab74b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.441123 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1396/2791] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:40:32 2022] DELETE /identity/v3/projects/c11ff59d05cd492ea27ecd1006f9f44f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:32.445091 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac408d5d-4d0e-47ae-981d-78f3f1ad8e0d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:32.446485 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ac408d5d-4d0e-47ae-981d-78f3f1ad8e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:32.459356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac408d5d-4d0e-47ae-981d-78f3f1ad8e0d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:32.515106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa311a27-d4de-42b6-8d6e-73e5b77aafd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:32.515771 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa311a27-d4de-42b6-8d6e-73e5b77aafd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:32.516055 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa311a27-d4de-42b6-8d6e-73e5b77aafd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:32.516331 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa311a27-d4de-42b6-8d6e-73e5b77aafd4 admin admin] PATH_INFO: `/v3/users/49b36a128fb440d28df0b7e39420b041` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:32.517470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa311a27-d4de-42b6-8d6e-73e5b77aafd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49b36a128fb440d28df0b7e39420b041)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:32.519389 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa311a27-d4de-42b6-8d6e-73e5b77aafd4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:32.567531 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-aa311a27-d4de-42b6-8d6e-73e5b77aafd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49b36a128fb440d28df0b7e39420b041'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.574527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-aa311a27-d4de-42b6-8d6e-73e5b77aafd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49b36a128fb440d28df0b7e39420b041'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.575188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-aa311a27-d4de-42b6-8d6e-73e5b77aafd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49b36a128fb440d28df0b7e39420b041'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.575824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-aa311a27-d4de-42b6-8d6e-73e5b77aafd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49b36a128fb440d28df0b7e39420b041'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.582958 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-aa311a27-d4de-42b6-8d6e-73e5b77aafd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49b36a128fb440d28df0b7e39420b041'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.600402 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1396/2792] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:40:32 2022] DELETE /identity/v3/users/49b36a128fb440d28df0b7e39420b041 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:32.772454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c431c8-d83e-4b4c-b469-1f29d3aab74b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:32.773933 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-39c431c8-d83e-4b4c-b469-1f29d3aab74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:32.787315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c431c8-d83e-4b4c-b469-1f29d3aab74b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:32.815718 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:32.851678 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a62894-af3a-45da-8d3a-ed31bf6366de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:32.852410 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a62894-af3a-45da-8d3a-ed31bf6366de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:32.852745 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a62894-af3a-45da-8d3a-ed31bf6366de admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:32.853086 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a62894-af3a-45da-8d3a-ed31bf6366de admin admin] PATH_INFO: `/v3/projects/6bd5aa0898834598ac30a381d4914cd9` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:32.854256 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a62894-af3a-45da-8d3a-ed31bf6366de admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bd5aa0898834598ac30a381d4914cd9)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:32.856840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a62894-af3a-45da-8d3a-ed31bf6366de admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:32.906135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b7a62894-af3a-45da-8d3a-ed31bf6366de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bd5aa0898834598ac30a381d4914cd9', 'request_id': 'req-b7a62894-af3a-45da-8d3a-ed31bf6366de'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.914156 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b7a62894-af3a-45da-8d3a-ed31bf6366de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bd5aa0898834598ac30a381d4914cd9', 'request_id': 'req-b7a62894-af3a-45da-8d3a-ed31bf6366de'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.921334 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b7a62894-af3a-45da-8d3a-ed31bf6366de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bd5aa0898834598ac30a381d4914cd9', 'request_id': 'req-b7a62894-af3a-45da-8d3a-ed31bf6366de'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.932568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b7a62894-af3a-45da-8d3a-ed31bf6366de admin admin] The token cache is being invalidate because project 6bd5aa0898834598ac30a381d4914cd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:40:32.932772 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b7a62894-af3a-45da-8d3a-ed31bf6366de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.933486 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-b7a62894-af3a-45da-8d3a-ed31bf6366de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:32.934966 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1397/2793] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:40:32 2022] DELETE /identity/v3/projects/6bd5aa0898834598ac30a381d4914cd9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:32.940127 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151340fc-615f-42bc-b80d-9da29677909e None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:32.940412 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151340fc-615f-42bc-b80d-9da29677909e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:32.940685 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151340fc-615f-42bc-b80d-9da29677909e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:32.960157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-151340fc-615f-42bc-b80d-9da29677909e None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:32.961360 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-151340fc-615f-42bc-b80d-9da29677909e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:32.970591 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:33.026004 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1397/2794] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:33.034559 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674662e-117c-459a-beb2-11675c68b58c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:33.034851 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674662e-117c-459a-beb2-11675c68b58c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:33.035128 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674662e-117c-459a-beb2-11675c68b58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:33.056991 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-c674662e-117c-459a-beb2-11675c68b58c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:33.058203 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-c674662e-117c-459a-beb2-11675c68b58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:33.066204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1398/2795] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:33.070838 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151340fc-615f-42bc-b80d-9da29677909e None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:33.080514 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151340fc-615f-42bc-b80d-9da29677909e None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:33.085136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff2803-7ec7-4dd6-88f4-f2033682c3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:33.085740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff2803-7ec7-4dd6-88f4-f2033682c3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:33.086023 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff2803-7ec7-4dd6-88f4-f2033682c3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:33.086296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff2803-7ec7-4dd6-88f4-f2033682c3e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:33.086934 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff2803-7ec7-4dd6-88f4-f2033682c3e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:40:33.087240 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff2803-7ec7-4dd6-88f4-f2033682c3e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:33.088968 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff2803-7ec7-4dd6-88f4-f2033682c3e5 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:33.100922 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1398/2796] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:40:33.105009 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674662e-117c-459a-beb2-11675c68b58c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:33.114657 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674662e-117c-459a-beb2-11675c68b58c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:33.119336 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c469f4c-cc09-4c9b-bad9-72a746008c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:33.120029 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c469f4c-cc09-4c9b-bad9-72a746008c09 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:33.120315 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c469f4c-cc09-4c9b-bad9-72a746008c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:33.120599 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c469f4c-cc09-4c9b-bad9-72a746008c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:33.121251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c469f4c-cc09-4c9b-bad9-72a746008c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:33.123132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c469f4c-cc09-4c9b-bad9-72a746008c09 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:33.150109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1399/2797] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:40:33 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:33.155041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff2803-7ec7-4dd6-88f4-f2033682c3e5 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:33.165259 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff2803-7ec7-4dd6-88f4-f2033682c3e5 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:33.221035 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8999d7c-f1fc-4924-b869-8e55b629c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:33.221565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8999d7c-f1fc-4924-b869-8e55b629c442 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:33.221850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8999d7c-f1fc-4924-b869-8e55b629c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:33.222151 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8999d7c-f1fc-4924-b869-8e55b629c442 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:33.222773 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8999d7c-f1fc-4924-b869-8e55b629c442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:33.224710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8999d7c-f1fc-4924-b869-8e55b629c442 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:33.245951 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1399/2798] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:40:33 2022] POST /identity/v3/users => generated 282 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:33.250103 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c469f4c-cc09-4c9b-bad9-72a746008c09 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:33.260429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c469f4c-cc09-4c9b-bad9-72a746008c09 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:33.265651 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc618ad6-82fe-4825-a766-62408f46ec02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:33.266255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc618ad6-82fe-4825-a766-62408f46ec02 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:33.266535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc618ad6-82fe-4825-a766-62408f46ec02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:33.266811 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc618ad6-82fe-4825-a766-62408f46ec02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:33.267451 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc618ad6-82fe-4825-a766-62408f46ec02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:33.269072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc618ad6-82fe-4825-a766-62408f46ec02 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:33.279417 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1400/2799] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:33.283242 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8999d7c-f1fc-4924-b869-8e55b629c442 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:33.293328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8999d7c-f1fc-4924-b869-8e55b629c442 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:33.297916 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d04340-98e6-428f-8928-e9dd5b44e915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:33.298505 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d04340-98e6-428f-8928-e9dd5b44e915 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:33.298790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d04340-98e6-428f-8928-e9dd5b44e915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:33.299065 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d04340-98e6-428f-8928-e9dd5b44e915 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:33.299789 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d04340-98e6-428f-8928-e9dd5b44e915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:33.301314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d04340-98e6-428f-8928-e9dd5b44e915 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:33.311842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1400/2800] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:33.315729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc618ad6-82fe-4825-a766-62408f46ec02 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:33.326071 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc618ad6-82fe-4825-a766-62408f46ec02 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:33.330622 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e3e3a-65cf-4e3e-9625-395effa7db84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:33.331251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e3e3a-65cf-4e3e-9625-395effa7db84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:33.331566 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e3e3a-65cf-4e3e-9625-395effa7db84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:33.331843 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e3e3a-65cf-4e3e-9625-395effa7db84 admin admin] PATH_INFO: `/v3/projects/053f1c13b3b948289a580636b919ea41/users/51ea01e494204f75baa6ce86070efeb9/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:33.346521 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e3e3a-65cf-4e3e-9625-395effa7db84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=053f1c13b3b948289a580636b919ea41, user_id=51ea01e494204f75baa6ce86070efeb9, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:33.349190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e3e3a-65cf-4e3e-9625-395effa7db84 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:33.361917 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1401/2801] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:40:33 2022] PUT /identity/v3/projects/053f1c13b3b948289a580636b919ea41/users/51ea01e494204f75baa6ce86070efeb9/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:33.372197 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c991c-1ef5-42cf-8353-e8e752a5b16f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:33.372499 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c991c-1ef5-42cf-8353-e8e752a5b16f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:33.372767 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c991c-1ef5-42cf-8353-e8e752a5b16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:33.400404 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d67c991c-1ef5-42cf-8353-e8e752a5b16f None None] MFA Rules not processed for user `51ea01e494204f75baa6ce86070efeb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:33.401595 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d67c991c-1ef5-42cf-8353-e8e752a5b16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:33.464172 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1401/2802] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:33.468727 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e3e3a-65cf-4e3e-9625-395effa7db84 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:33.520146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:33.532615 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e3e3a-65cf-4e3e-9625-395effa7db84 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:33.537988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d16035-b505-49af-ac78-3c3451f8e777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:33.538595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d16035-b505-49af-ac78-3c3451f8e777 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:33.538875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d16035-b505-49af-ac78-3c3451f8e777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:33.539147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d16035-b505-49af-ac78-3c3451f8e777 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:33.539820 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d16035-b505-49af-ac78-3c3451f8e777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:33.541704 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d16035-b505-49af-ac78-3c3451f8e777 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:33.567633 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1402/2803] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:40:33 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:40:33.571654 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c991c-1ef5-42cf-8353-e8e752a5b16f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:33.582060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c991c-1ef5-42cf-8353-e8e752a5b16f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:33.586550 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:33.639629 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cace01-6f0d-43ca-8f09-cfd0ca4b8402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:33.640232 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cace01-6f0d-43ca-8f09-cfd0ca4b8402 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:33.640509 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cace01-6f0d-43ca-8f09-cfd0ca4b8402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:33.640781 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cace01-6f0d-43ca-8f09-cfd0ca4b8402 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:33.641408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cace01-6f0d-43ca-8f09-cfd0ca4b8402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:33.643307 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cace01-6f0d-43ca-8f09-cfd0ca4b8402 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:33.664855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1402/2804] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:40:33 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:33.669830 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d16035-b505-49af-ac78-3c3451f8e777 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:33.679862 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d16035-b505-49af-ac78-3c3451f8e777 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:33.684408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15de45b-d053-49ec-b42f-164add708a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:33.684993 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15de45b-d053-49ec-b42f-164add708a30 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:33.685272 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15de45b-d053-49ec-b42f-164add708a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:33.685543 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15de45b-d053-49ec-b42f-164add708a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:33.686174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15de45b-d053-49ec-b42f-164add708a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:33.687792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15de45b-d053-49ec-b42f-164add708a30 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:33.697419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1403/2805] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:33.701545 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cace01-6f0d-43ca-8f09-cfd0ca4b8402 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:33.711470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cace01-6f0d-43ca-8f09-cfd0ca4b8402 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:33.716679 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda1fdd-e244-439f-9691-0728875f4c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:33.717339 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda1fdd-e244-439f-9691-0728875f4c6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:33.717601 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda1fdd-e244-439f-9691-0728875f4c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:33.717874 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda1fdd-e244-439f-9691-0728875f4c6d admin admin] PATH_INFO: `/v3/projects/f1ef7559c0bd45c0a3b47e20fb4b75de/users/405c83b7d20a4a4ebf9e481f55b3324a/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:33.733056 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda1fdd-e244-439f-9691-0728875f4c6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ef7559c0bd45c0a3b47e20fb4b75de, user_id=405c83b7d20a4a4ebf9e481f55b3324a, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:33.735660 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda1fdd-e244-439f-9691-0728875f4c6d admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:33.748264 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1403/2806] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:40:33 2022] PUT /identity/v3/projects/f1ef7559c0bd45c0a3b47e20fb4b75de/users/405c83b7d20a4a4ebf9e481f55b3324a/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:33.752093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15de45b-d053-49ec-b42f-164add708a30 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:33.761654 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15de45b-d053-49ec-b42f-164add708a30 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:33.817897 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcd295-9193-4a20-a91f-5425ff9c4b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:33.818547 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd295-9193-4a20-a91f-5425ff9c4b98 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:33.818827 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd295-9193-4a20-a91f-5425ff9c4b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:33.819099 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcd295-9193-4a20-a91f-5425ff9c4b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:33.819765 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcd295-9193-4a20-a91f-5425ff9c4b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:33.821345 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcd295-9193-4a20-a91f-5425ff9c4b98 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:33.830857 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1404/2807] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:33.835045 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda1fdd-e244-439f-9691-0728875f4c6d admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:33.845755 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda1fdd-e244-439f-9691-0728875f4c6d admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:33.850337 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cec11-b3e6-4998-9a85-a6919b0826ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:33.850979 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cec11-b3e6-4998-9a85-a6919b0826ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:33.851279 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cec11-b3e6-4998-9a85-a6919b0826ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:33.851584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cec11-b3e6-4998-9a85-a6919b0826ce admin admin] PATH_INFO: `/v3/projects/f1ef7559c0bd45c0a3b47e20fb4b75de/users/405c83b7d20a4a4ebf9e481f55b3324a/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:33.853469 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4cec11-b3e6-4998-9a85-a6919b0826ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ef7559c0bd45c0a3b47e20fb4b75de, user_id=405c83b7d20a4a4ebf9e481f55b3324a, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:33.856742 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4cec11-b3e6-4998-9a85-a6919b0826ce admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:33.867145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1404/2808] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:40:33 2022] PUT /identity/v3/projects/f1ef7559c0bd45c0a3b47e20fb4b75de/users/405c83b7d20a4a4ebf9e481f55b3324a/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:33.878392 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec108-d9b2-4054-8352-79ab2c6e2f30 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:33.878681 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec108-d9b2-4054-8352-79ab2c6e2f30 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:33.878986 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec108-d9b2-4054-8352-79ab2c6e2f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:33.908153 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-214ec108-d9b2-4054-8352-79ab2c6e2f30 None None] MFA Rules not processed for user `405c83b7d20a4a4ebf9e481f55b3324a`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:33.909387 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-214ec108-d9b2-4054-8352-79ab2c6e2f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:33.975614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1405/2809] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:33.987312 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cec11-b3e6-4998-9a85-a6919b0826ce admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:33.989110 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6a4cec11-b3e6-4998-9a85-a6919b0826ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:34.045860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cec11-b3e6-4998-9a85-a6919b0826ce admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:34.049518 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86ca21-5e0b-4c7f-b21e-69fdf22acdbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:34.050107 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86ca21-5e0b-4c7f-b21e-69fdf22acdbb service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:34.050383 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86ca21-5e0b-4c7f-b21e-69fdf22acdbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:34.050652 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86ca21-5e0b-4c7f-b21e-69fdf22acdbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:34.059926 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86ca21-5e0b-4c7f-b21e-69fdf22acdbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:34.061328 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86ca21-5e0b-4c7f-b21e-69fdf22acdbb service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:34.070457 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1405/2810] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:40:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:40:34.469854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ec108-d9b2-4054-8352-79ab2c6e2f30 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:34.516647 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:34.531454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ec108-d9b2-4054-8352-79ab2c6e2f30 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:34.536302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53305cb0-8a79-4c43-bbad-c9e5b2e4cbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:34.536932 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53305cb0-8a79-4c43-bbad-c9e5b2e4cbce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:34.537212 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53305cb0-8a79-4c43-bbad-c9e5b2e4cbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:34.537483 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53305cb0-8a79-4c43-bbad-c9e5b2e4cbce admin admin] PATH_INFO: `/v3/users/51ea01e494204f75baa6ce86070efeb9` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:34.538598 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53305cb0-8a79-4c43-bbad-c9e5b2e4cbce admin admin] RBAC: Authorizing `identity:delete_user(user_id=51ea01e494204f75baa6ce86070efeb9)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:34.540567 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53305cb0-8a79-4c43-bbad-c9e5b2e4cbce admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:34.585025 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-53305cb0-8a79-4c43-bbad-c9e5b2e4cbce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51ea01e494204f75baa6ce86070efeb9'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:34.592311 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-53305cb0-8a79-4c43-bbad-c9e5b2e4cbce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51ea01e494204f75baa6ce86070efeb9'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:34.592950 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-53305cb0-8a79-4c43-bbad-c9e5b2e4cbce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51ea01e494204f75baa6ce86070efeb9'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:34.593549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-53305cb0-8a79-4c43-bbad-c9e5b2e4cbce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51ea01e494204f75baa6ce86070efeb9'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:34.600672 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-53305cb0-8a79-4c43-bbad-c9e5b2e4cbce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51ea01e494204f75baa6ce86070efeb9'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:34.618187 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1406/2811] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:40:34 2022] DELETE /identity/v3/users/51ea01e494204f75baa6ce86070efeb9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:34.622662 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9100da1-90ab-4732-b731-c33022c6bc82 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:34.623015 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9100da1-90ab-4732-b731-c33022c6bc82 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:34.623297 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9100da1-90ab-4732-b731-c33022c6bc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:34.642188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-d9100da1-90ab-4732-b731-c33022c6bc82 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:34.643389 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d9100da1-90ab-4732-b731-c33022c6bc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:34.701364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1406/2812] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:34.711173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53305cb0-8a79-4c43-bbad-c9e5b2e4cbce admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:34.712646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-53305cb0-8a79-4c43-bbad-c9e5b2e4cbce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:34.768826 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53305cb0-8a79-4c43-bbad-c9e5b2e4cbce admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:34.772434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c53781-3eda-41ff-a5a7-2fa7a5ccd507 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:34.773033 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c53781-3eda-41ff-a5a7-2fa7a5ccd507 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:34.773310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c53781-3eda-41ff-a5a7-2fa7a5ccd507 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:34.773580 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c53781-3eda-41ff-a5a7-2fa7a5ccd507 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:34.782942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c53781-3eda-41ff-a5a7-2fa7a5ccd507 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:34.784457 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c53781-3eda-41ff-a5a7-2fa7a5ccd507 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:34.793911 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1407/2813] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:40:35.053976 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9100da1-90ab-4732-b731-c33022c6bc82 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:35.055454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-d9100da1-90ab-4732-b731-c33022c6bc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:35.063845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:35.067860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9100da1-90ab-4732-b731-c33022c6bc82 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:35.073630 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:35.074296 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:35.074576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:35.074853 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7 admin admin] PATH_INFO: `/v3/projects/053f1c13b3b948289a580636b919ea41` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:35.076017 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=053f1c13b3b948289a580636b919ea41)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:35.077924 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:35.125155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '053f1c13b3b948289a580636b919ea41', 'request_id': 'req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:35.131810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '053f1c13b3b948289a580636b919ea41', 'request_id': 'req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:35.139931 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '053f1c13b3b948289a580636b919ea41', 'request_id': 'req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:35.155120 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7 admin admin] The token cache is being invalidate because project 053f1c13b3b948289a580636b919ea41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:40:35.155408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:35.156093 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:35.157591 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1407/2814] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:40:35 2022] DELETE /identity/v3/projects/053f1c13b3b948289a580636b919ea41 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:35.162028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c53781-3eda-41ff-a5a7-2fa7a5ccd507 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:35.172226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c53781-3eda-41ff-a5a7-2fa7a5ccd507 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:35.226480 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911eaca3-2834-427c-833b-65eef11b72e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:35.227099 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eaca3-2834-427c-833b-65eef11b72e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:35.227376 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eaca3-2834-427c-833b-65eef11b72e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:35.227683 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eaca3-2834-427c-833b-65eef11b72e6 admin admin] PATH_INFO: `/v3/users/405c83b7d20a4a4ebf9e481f55b3324a` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:35.228779 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911eaca3-2834-427c-833b-65eef11b72e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=405c83b7d20a4a4ebf9e481f55b3324a)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:35.231250 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911eaca3-2834-427c-833b-65eef11b72e6 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:35.243357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:35.277757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-911eaca3-2834-427c-833b-65eef11b72e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '405c83b7d20a4a4ebf9e481f55b3324a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:35.284722 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-911eaca3-2834-427c-833b-65eef11b72e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '405c83b7d20a4a4ebf9e481f55b3324a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:35.285401 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-911eaca3-2834-427c-833b-65eef11b72e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '405c83b7d20a4a4ebf9e481f55b3324a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:35.286267 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-911eaca3-2834-427c-833b-65eef11b72e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '405c83b7d20a4a4ebf9e481f55b3324a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:35.294215 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-911eaca3-2834-427c-833b-65eef11b72e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '405c83b7d20a4a4ebf9e481f55b3324a'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:35.311405 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1408/2815] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:40:35 2022] DELETE /identity/v3/users/405c83b7d20a4a4ebf9e481f55b3324a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:35.578235 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:35.579689 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:35.594051 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3abbb2-a310-4cfa-b464-8a7ba2a6bec7 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:35.650917 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064f9e4-0648-42b4-a327-754f29bd4567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:35.651575 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064f9e4-0648-42b4-a327-754f29bd4567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:35.651860 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064f9e4-0648-42b4-a327-754f29bd4567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:35.652134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064f9e4-0648-42b4-a327-754f29bd4567 admin admin] PATH_INFO: `/v3/projects/f1ef7559c0bd45c0a3b47e20fb4b75de` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:35.653249 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064f9e4-0648-42b4-a327-754f29bd4567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1ef7559c0bd45c0a3b47e20fb4b75de)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:35.655148 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064f9e4-0648-42b4-a327-754f29bd4567 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:35.701198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9064f9e4-0648-42b4-a327-754f29bd4567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1ef7559c0bd45c0a3b47e20fb4b75de', 'request_id': 'req-9064f9e4-0648-42b4-a327-754f29bd4567'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:35.708028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9064f9e4-0648-42b4-a327-754f29bd4567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1ef7559c0bd45c0a3b47e20fb4b75de', 'request_id': 'req-9064f9e4-0648-42b4-a327-754f29bd4567'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:35.714210 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9064f9e4-0648-42b4-a327-754f29bd4567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1ef7559c0bd45c0a3b47e20fb4b75de', 'request_id': 'req-9064f9e4-0648-42b4-a327-754f29bd4567'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:35.724686 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9064f9e4-0648-42b4-a327-754f29bd4567 admin admin] The token cache is being invalidate because project f1ef7559c0bd45c0a3b47e20fb4b75de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:40:35.724973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9064f9e4-0648-42b4-a327-754f29bd4567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:35.725613 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-9064f9e4-0648-42b4-a327-754f29bd4567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:35.727099 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1408/2816] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:40:35 2022] DELETE /identity/v3/projects/f1ef7559c0bd45c0a3b47e20fb4b75de => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:35.732467 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8db3e2-245c-4478-ba93-9bb8dcbc6de3 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:35.732976 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8db3e2-245c-4478-ba93-9bb8dcbc6de3 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:35.733092 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8db3e2-245c-4478-ba93-9bb8dcbc6de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:35.752704 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-1b8db3e2-245c-4478-ba93-9bb8dcbc6de3 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:35.753883 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-1b8db3e2-245c-4478-ba93-9bb8dcbc6de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:35.813720 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1409/2817] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:35.821523 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db192d-4f8b-4e8e-8785-37e287f7e62c None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:35.821753 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db192d-4f8b-4e8e-8785-37e287f7e62c None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:35.822031 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db192d-4f8b-4e8e-8785-37e287f7e62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:35.842030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-96db192d-4f8b-4e8e-8785-37e287f7e62c None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:35.843236 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-96db192d-4f8b-4e8e-8785-37e287f7e62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:35.851272 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1409/2818] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:35.856088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8db3e2-245c-4478-ba93-9bb8dcbc6de3 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:35.865641 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8db3e2-245c-4478-ba93-9bb8dcbc6de3 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:35.870276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0d388-ba4f-44ce-9a28-9b6c41614062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:35.870874 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0d388-ba4f-44ce-9a28-9b6c41614062 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:35.871157 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0d388-ba4f-44ce-9a28-9b6c41614062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:35.871434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0d388-ba4f-44ce-9a28-9b6c41614062 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:35.872120 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0d388-ba4f-44ce-9a28-9b6c41614062 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:40:35.872428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0d388-ba4f-44ce-9a28-9b6c41614062 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:35.874023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0d388-ba4f-44ce-9a28-9b6c41614062 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:35.885302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1410/2819] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:40:35.889584 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db192d-4f8b-4e8e-8785-37e287f7e62c None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:35.898710 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db192d-4f8b-4e8e-8785-37e287f7e62c None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:35.903910 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d6c05-eb73-49c5-a29c-84a6814bf1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:35.904500 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d6c05-eb73-49c5-a29c-84a6814bf1c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:35.904780 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d6c05-eb73-49c5-a29c-84a6814bf1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:35.905055 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d6c05-eb73-49c5-a29c-84a6814bf1c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:35.905677 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d6c05-eb73-49c5-a29c-84a6814bf1c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:35.907587 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d6c05-eb73-49c5-a29c-84a6814bf1c3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:35.917085 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:35.935364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1410/2820] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:40:35 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:35.939318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0d388-ba4f-44ce-9a28-9b6c41614062 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:35.949199 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0d388-ba4f-44ce-9a28-9b6c41614062 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:36.004389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fa0ce-3fa4-43af-9136-9db8029059c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:36.005016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fa0ce-3fa4-43af-9136-9db8029059c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:36.005302 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fa0ce-3fa4-43af-9136-9db8029059c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:36.005555 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fa0ce-3fa4-43af-9136-9db8029059c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:36.006184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fa0ce-3fa4-43af-9136-9db8029059c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:36.008131 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fa0ce-3fa4-43af-9136-9db8029059c4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:36.029078 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1411/2821] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:40:35 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:36.033125 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d6c05-eb73-49c5-a29c-84a6814bf1c3 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:36.044321 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d6c05-eb73-49c5-a29c-84a6814bf1c3 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:36.048936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06066307-6fe4-4e58-9194-0333960d1fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:36.049528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06066307-6fe4-4e58-9194-0333960d1fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:36.049830 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06066307-6fe4-4e58-9194-0333960d1fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:36.050088 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06066307-6fe4-4e58-9194-0333960d1fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:36.050727 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06066307-6fe4-4e58-9194-0333960d1fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:36.052336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06066307-6fe4-4e58-9194-0333960d1fc3 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:36.062082 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1411/2822] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:36.065709 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fa0ce-3fa4-43af-9136-9db8029059c4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:36.075525 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fa0ce-3fa4-43af-9136-9db8029059c4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:36.080373 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8945d6-4272-427e-b2d0-44c516b0e3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:36.080973 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8945d6-4272-427e-b2d0-44c516b0e3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:36.081255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8945d6-4272-427e-b2d0-44c516b0e3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:36.081532 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8945d6-4272-427e-b2d0-44c516b0e3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:36.082171 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8945d6-4272-427e-b2d0-44c516b0e3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:36.083790 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8945d6-4272-427e-b2d0-44c516b0e3c8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:36.085073 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:36.098407 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1412/2823] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:36.099801 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06066307-6fe4-4e58-9194-0333960d1fc3 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:36.110202 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06066307-6fe4-4e58-9194-0333960d1fc3 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:36.114750 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127c6a2-d659-414f-b1a0-60b6248431f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:36.115419 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127c6a2-d659-414f-b1a0-60b6248431f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:36.115769 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127c6a2-d659-414f-b1a0-60b6248431f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:36.116149 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127c6a2-d659-414f-b1a0-60b6248431f7 admin admin] PATH_INFO: `/v3/projects/67fb9e9ca8c94d9583ea2c948d035f67/users/4ed5e32dfd5f4a34aaad77332e9f3adc/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:36.130988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127c6a2-d659-414f-b1a0-60b6248431f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67fb9e9ca8c94d9583ea2c948d035f67, user_id=4ed5e32dfd5f4a34aaad77332e9f3adc, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:36.133676 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127c6a2-d659-414f-b1a0-60b6248431f7 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:36.146935 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1412/2824] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:40:36 2022] PUT /identity/v3/projects/67fb9e9ca8c94d9583ea2c948d035f67/users/4ed5e32dfd5f4a34aaad77332e9f3adc/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:36.158994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0a038-6533-4efd-ab79-ad8057ec114b None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:36.159286 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0a038-6533-4efd-ab79-ad8057ec114b None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:36.159605 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0a038-6533-4efd-ab79-ad8057ec114b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:36.188166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-d6b0a038-6533-4efd-ab79-ad8057ec114b None None] MFA Rules not processed for user `4ed5e32dfd5f4a34aaad77332e9f3adc`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:36.189405 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d6b0a038-6533-4efd-ab79-ad8057ec114b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:36.251777 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1413/2825] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:36.256183 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127c6a2-d659-414f-b1a0-60b6248431f7 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:36.317990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127c6a2-d659-414f-b1a0-60b6248431f7 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:36.322850 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6f780-fb7f-4d0e-b8b0-a523c32ab4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:36.323441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6f780-fb7f-4d0e-b8b0-a523c32ab4ba admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:36.323751 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6f780-fb7f-4d0e-b8b0-a523c32ab4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:36.324523 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6f780-fb7f-4d0e-b8b0-a523c32ab4ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:36.325134 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6f780-fb7f-4d0e-b8b0-a523c32ab4ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:36.327059 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6f780-fb7f-4d0e-b8b0-a523c32ab4ba admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:36.352884 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1413/2826] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:40:36 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:36.356905 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0a038-6533-4efd-ab79-ad8057ec114b None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:36.367343 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0a038-6533-4efd-ab79-ad8057ec114b None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:36.423702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c29212-f8cc-422c-9dc6-1e2ec85cc099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:36.424312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c29212-f8cc-422c-9dc6-1e2ec85cc099 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:36.424578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c29212-f8cc-422c-9dc6-1e2ec85cc099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:36.424849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c29212-f8cc-422c-9dc6-1e2ec85cc099 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:36.425472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c29212-f8cc-422c-9dc6-1e2ec85cc099 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:36.427369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c29212-f8cc-422c-9dc6-1e2ec85cc099 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:36.448296 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1414/2827] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:40:36 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:36.452394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6f780-fb7f-4d0e-b8b0-a523c32ab4ba admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:36.462616 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6f780-fb7f-4d0e-b8b0-a523c32ab4ba admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:36.468422 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c06f55-b7fd-49e6-be8d-8d80b3cdbfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:36.469261 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c06f55-b7fd-49e6-be8d-8d80b3cdbfbc admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:36.469692 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c06f55-b7fd-49e6-be8d-8d80b3cdbfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:36.470262 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c06f55-b7fd-49e6-be8d-8d80b3cdbfbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:36.470906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c06f55-b7fd-49e6-be8d-8d80b3cdbfbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:36.472526 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c06f55-b7fd-49e6-be8d-8d80b3cdbfbc admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:36.483824 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1414/2828] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:36.487822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c29212-f8cc-422c-9dc6-1e2ec85cc099 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:36.498207 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c29212-f8cc-422c-9dc6-1e2ec85cc099 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:36.503000 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cc71b-94af-4160-bec1-a0f1df2a5b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:36.503693 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cc71b-94af-4160-bec1-a0f1df2a5b10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:36.504051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cc71b-94af-4160-bec1-a0f1df2a5b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:36.504328 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cc71b-94af-4160-bec1-a0f1df2a5b10 admin admin] PATH_INFO: `/v3/projects/7812275daf624fb69de3d90e7c5e770c/users/a683e7a75a0f4d3abc09b4e113d5db2d/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:36.518931 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cc71b-94af-4160-bec1-a0f1df2a5b10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7812275daf624fb69de3d90e7c5e770c, user_id=a683e7a75a0f4d3abc09b4e113d5db2d, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:36.521638 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cc71b-94af-4160-bec1-a0f1df2a5b10 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:36.535022 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1415/2829] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:40:36 2022] PUT /identity/v3/projects/7812275daf624fb69de3d90e7c5e770c/users/a683e7a75a0f4d3abc09b4e113d5db2d/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:36.539106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c06f55-b7fd-49e6-be8d-8d80b3cdbfbc admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:36.549434 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c06f55-b7fd-49e6-be8d-8d80b3cdbfbc admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:36.603300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5a6fb-ed8c-4867-99ac-81750e37beac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:36.603926 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5a6fb-ed8c-4867-99ac-81750e37beac admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:36.604203 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5a6fb-ed8c-4867-99ac-81750e37beac admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:36.604470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5a6fb-ed8c-4867-99ac-81750e37beac admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:36.605201 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5a6fb-ed8c-4867-99ac-81750e37beac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:36.607739 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5a6fb-ed8c-4867-99ac-81750e37beac admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:36.617148 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1415/2830] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:36.621750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cc71b-94af-4160-bec1-a0f1df2a5b10 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:36.631873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cc71b-94af-4160-bec1-a0f1df2a5b10 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:36.637094 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec320e-72cb-4119-8458-219e8f7ef720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:36.637737 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec320e-72cb-4119-8458-219e8f7ef720 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:36.638020 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec320e-72cb-4119-8458-219e8f7ef720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:36.638295 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec320e-72cb-4119-8458-219e8f7ef720 admin admin] PATH_INFO: `/v3/projects/7812275daf624fb69de3d90e7c5e770c/users/a683e7a75a0f4d3abc09b4e113d5db2d/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:36.640250 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec320e-72cb-4119-8458-219e8f7ef720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7812275daf624fb69de3d90e7c5e770c, user_id=a683e7a75a0f4d3abc09b4e113d5db2d, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:36.642860 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec320e-72cb-4119-8458-219e8f7ef720 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:36.653010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1416/2831] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:40:36 2022] PUT /identity/v3/projects/7812275daf624fb69de3d90e7c5e770c/users/a683e7a75a0f4d3abc09b4e113d5db2d/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:36.714650 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6d916-66bb-4a07-b36e-4a5de05fc8c1 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:36.714938 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6d916-66bb-4a07-b36e-4a5de05fc8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:36.715214 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6d916-66bb-4a07-b36e-4a5de05fc8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:36.742464 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-33a6d916-66bb-4a07-b36e-4a5de05fc8c1 None None] MFA Rules not processed for user `a683e7a75a0f4d3abc09b4e113d5db2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:36.743713 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-33a6d916-66bb-4a07-b36e-4a5de05fc8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:36.755899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:36.811822 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1416/2832] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:36.822994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec320e-72cb-4119-8458-219e8f7ef720 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:36.825326 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ecec320e-72cb-4119-8458-219e8f7ef720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:36.866216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:36.882317 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec320e-72cb-4119-8458-219e8f7ef720 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:36.885947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48a58b-d287-4a59-90f6-66baf4b460bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:36.887158 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48a58b-d287-4a59-90f6-66baf4b460bd service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:36.887433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48a58b-d287-4a59-90f6-66baf4b460bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:36.887734 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48a58b-d287-4a59-90f6-66baf4b460bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:36.896464 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48a58b-d287-4a59-90f6-66baf4b460bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:36.897901 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48a58b-d287-4a59-90f6-66baf4b460bd service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:36.907016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1417/2833] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:40:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:40:36.952090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6d916-66bb-4a07-b36e-4a5de05fc8c1 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:36.953477 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-33a6d916-66bb-4a07-b36e-4a5de05fc8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:36.965458 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6d916-66bb-4a07-b36e-4a5de05fc8c1 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:36.969030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee716c-5db8-4e0f-9730-fa01ccdcbf09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:36.969624 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee716c-5db8-4e0f-9730-fa01ccdcbf09 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:36.969907 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee716c-5db8-4e0f-9730-fa01ccdcbf09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:36.970181 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee716c-5db8-4e0f-9730-fa01ccdcbf09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:37.027528 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee716c-5db8-4e0f-9730-fa01ccdcbf09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:37.029097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee716c-5db8-4e0f-9730-fa01ccdcbf09 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:37.038033 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1417/2834] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:40:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:40:40.306994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48a58b-d287-4a59-90f6-66baf4b460bd service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:40.387444 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48a58b-d287-4a59-90f6-66baf4b460bd service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:40.391997 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016eceef-ead0-4dd0-a627-f2a3ef62b9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:40.392614 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eceef-ead0-4dd0-a627-f2a3ef62b9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:40.392894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eceef-ead0-4dd0-a627-f2a3ef62b9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:40.393173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eceef-ead0-4dd0-a627-f2a3ef62b9a8 admin admin] PATH_INFO: `/v3/users/4ed5e32dfd5f4a34aaad77332e9f3adc` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:40.394276 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016eceef-ead0-4dd0-a627-f2a3ef62b9a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ed5e32dfd5f4a34aaad77332e9f3adc)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:40.396218 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016eceef-ead0-4dd0-a627-f2a3ef62b9a8 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:40.442048 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-016eceef-ead0-4dd0-a627-f2a3ef62b9a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ed5e32dfd5f4a34aaad77332e9f3adc'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:40.448889 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-016eceef-ead0-4dd0-a627-f2a3ef62b9a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ed5e32dfd5f4a34aaad77332e9f3adc'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:40.450034 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-016eceef-ead0-4dd0-a627-f2a3ef62b9a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ed5e32dfd5f4a34aaad77332e9f3adc'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:40.450642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-016eceef-ead0-4dd0-a627-f2a3ef62b9a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ed5e32dfd5f4a34aaad77332e9f3adc'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:40.457885 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-016eceef-ead0-4dd0-a627-f2a3ef62b9a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ed5e32dfd5f4a34aaad77332e9f3adc'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:40.474647 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1418/2835] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:40:40 2022] DELETE /identity/v3/users/4ed5e32dfd5f4a34aaad77332e9f3adc => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:40.479177 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de2649-24b4-4e88-88f6-44114050cc64 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:40.479694 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de2649-24b4-4e88-88f6-44114050cc64 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:40.479807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de2649-24b4-4e88-88f6-44114050cc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:40.510000 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-95de2649-24b4-4e88-88f6-44114050cc64 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:40.511210 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-95de2649-24b4-4e88-88f6-44114050cc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:40.568578 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1418/2836] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:40.578800 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016eceef-ead0-4dd0-a627-f2a3ef62b9a8 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:40.580237 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-016eceef-ead0-4dd0-a627-f2a3ef62b9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:40.624766 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:40.659291 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016eceef-ead0-4dd0-a627-f2a3ef62b9a8 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:40.663255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969e9ac-a02e-4c74-a179-49f859ddaa25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:40.663892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969e9ac-a02e-4c74-a179-49f859ddaa25 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:40.664177 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969e9ac-a02e-4c74-a179-49f859ddaa25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:40.664450 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969e9ac-a02e-4c74-a179-49f859ddaa25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:40.673819 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969e9ac-a02e-4c74-a179-49f859ddaa25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:40.675245 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969e9ac-a02e-4c74-a179-49f859ddaa25 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:40.685219 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1419/2837] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:40:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:40:41.187008 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de2649-24b4-4e88-88f6-44114050cc64 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:41.188481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-95de2649-24b4-4e88-88f6-44114050cc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:41.200391 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de2649-24b4-4e88-88f6-44114050cc64 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:41.205048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6cf30-2e64-4f51-b603-7924adaba530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:41.205678 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6cf30-2e64-4f51-b603-7924adaba530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:41.205966 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6cf30-2e64-4f51-b603-7924adaba530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:41.206242 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6cf30-2e64-4f51-b603-7924adaba530 admin admin] PATH_INFO: `/v3/projects/67fb9e9ca8c94d9583ea2c948d035f67` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:41.207557 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6cf30-2e64-4f51-b603-7924adaba530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67fb9e9ca8c94d9583ea2c948d035f67)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:41.209806 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6cf30-2e64-4f51-b603-7924adaba530 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:41.246379 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:41.261155 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3de6cf30-2e64-4f51-b603-7924adaba530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67fb9e9ca8c94d9583ea2c948d035f67', 'request_id': 'req-3de6cf30-2e64-4f51-b603-7924adaba530'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:41.268646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3de6cf30-2e64-4f51-b603-7924adaba530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67fb9e9ca8c94d9583ea2c948d035f67', 'request_id': 'req-3de6cf30-2e64-4f51-b603-7924adaba530'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:41.275036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3de6cf30-2e64-4f51-b603-7924adaba530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67fb9e9ca8c94d9583ea2c948d035f67', 'request_id': 'req-3de6cf30-2e64-4f51-b603-7924adaba530'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:41.286081 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3de6cf30-2e64-4f51-b603-7924adaba530 admin admin] The token cache is being invalidate because project 67fb9e9ca8c94d9583ea2c948d035f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:40:41.286373 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3de6cf30-2e64-4f51-b603-7924adaba530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:41.287004 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3de6cf30-2e64-4f51-b603-7924adaba530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:41.288538 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1419/2838] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:40:41 2022] DELETE /identity/v3/projects/67fb9e9ca8c94d9583ea2c948d035f67 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:41.293119 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969e9ac-a02e-4c74-a179-49f859ddaa25 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:41.302578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969e9ac-a02e-4c74-a179-49f859ddaa25 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:41.357615 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ce074-b4d3-40d2-92af-819032adc432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:41.358268 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ce074-b4d3-40d2-92af-819032adc432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:41.358549 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ce074-b4d3-40d2-92af-819032adc432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:41.358829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ce074-b4d3-40d2-92af-819032adc432 admin admin] PATH_INFO: `/v3/users/a683e7a75a0f4d3abc09b4e113d5db2d` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:41.359988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ce074-b4d3-40d2-92af-819032adc432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a683e7a75a0f4d3abc09b4e113d5db2d)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:41.361892 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ce074-b4d3-40d2-92af-819032adc432 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:41.406840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3f3ce074-b4d3-40d2-92af-819032adc432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a683e7a75a0f4d3abc09b4e113d5db2d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:41.414036 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3f3ce074-b4d3-40d2-92af-819032adc432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a683e7a75a0f4d3abc09b4e113d5db2d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:41.414684 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3f3ce074-b4d3-40d2-92af-819032adc432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a683e7a75a0f4d3abc09b4e113d5db2d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:41.415288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3f3ce074-b4d3-40d2-92af-819032adc432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a683e7a75a0f4d3abc09b4e113d5db2d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:41.423615 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-3f3ce074-b4d3-40d2-92af-819032adc432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a683e7a75a0f4d3abc09b4e113d5db2d'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:41.441446 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1420/2839] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:40:41 2022] DELETE /identity/v3/users/a683e7a75a0f4d3abc09b4e113d5db2d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:41.702404 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6cf30-2e64-4f51-b603-7924adaba530 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:41.704048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3de6cf30-2e64-4f51-b603-7924adaba530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:41.717987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6cf30-2e64-4f51-b603-7924adaba530 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:41.777768 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbb5de-f8d0-4f28-8a66-a7b338b3f1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:41.778394 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbb5de-f8d0-4f28-8a66-a7b338b3f1d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:41.778675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbb5de-f8d0-4f28-8a66-a7b338b3f1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:41.778947 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbb5de-f8d0-4f28-8a66-a7b338b3f1d1 admin admin] PATH_INFO: `/v3/projects/7812275daf624fb69de3d90e7c5e770c` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:41.780087 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacbb5de-f8d0-4f28-8a66-a7b338b3f1d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7812275daf624fb69de3d90e7c5e770c)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:41.781965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacbb5de-f8d0-4f28-8a66-a7b338b3f1d1 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:41.828096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bacbb5de-f8d0-4f28-8a66-a7b338b3f1d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7812275daf624fb69de3d90e7c5e770c', 'request_id': 'req-bacbb5de-f8d0-4f28-8a66-a7b338b3f1d1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:41.835371 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bacbb5de-f8d0-4f28-8a66-a7b338b3f1d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7812275daf624fb69de3d90e7c5e770c', 'request_id': 'req-bacbb5de-f8d0-4f28-8a66-a7b338b3f1d1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:41.841905 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bacbb5de-f8d0-4f28-8a66-a7b338b3f1d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7812275daf624fb69de3d90e7c5e770c', 'request_id': 'req-bacbb5de-f8d0-4f28-8a66-a7b338b3f1d1'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:41.852702 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bacbb5de-f8d0-4f28-8a66-a7b338b3f1d1 admin admin] The token cache is being invalidate because project 7812275daf624fb69de3d90e7c5e770c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:40:41.852987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bacbb5de-f8d0-4f28-8a66-a7b338b3f1d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:41.853614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-bacbb5de-f8d0-4f28-8a66-a7b338b3f1d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:41.855756 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1420/2840] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:40:41 2022] DELETE /identity/v3/projects/7812275daf624fb69de3d90e7c5e770c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:41.860629 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64177d7f-c70b-46a0-b460-280c19c62a85 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:41.860916 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64177d7f-c70b-46a0-b460-280c19c62a85 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:41.861193 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64177d7f-c70b-46a0-b460-280c19c62a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:41.917428 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-64177d7f-c70b-46a0-b460-280c19c62a85 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:41.919200 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-64177d7f-c70b-46a0-b460-280c19c62a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:41.927323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:41.978210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1421/2841] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:41.986325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264d9af-aa8a-492f-be2d-03e5926d88bb None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:41.986618 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264d9af-aa8a-492f-be2d-03e5926d88bb None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:41.986895 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264d9af-aa8a-492f-be2d-03e5926d88bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:42.006342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-7264d9af-aa8a-492f-be2d-03e5926d88bb None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:42.007574 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7264d9af-aa8a-492f-be2d-03e5926d88bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:42.015431 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1421/2842] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:42.019459 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64177d7f-c70b-46a0-b460-280c19c62a85 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:42.029496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64177d7f-c70b-46a0-b460-280c19c62a85 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:42.034088 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc517b-8e38-4544-998c-7111f199f3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:42.034682 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc517b-8e38-4544-998c-7111f199f3cc admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:42.034962 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc517b-8e38-4544-998c-7111f199f3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:42.035235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc517b-8e38-4544-998c-7111f199f3cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:42.036166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc517b-8e38-4544-998c-7111f199f3cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:40:42.036471 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc517b-8e38-4544-998c-7111f199f3cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:42.038055 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc517b-8e38-4544-998c-7111f199f3cc admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:42.050089 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1422/2843] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:40:42.054309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264d9af-aa8a-492f-be2d-03e5926d88bb None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:42.063491 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264d9af-aa8a-492f-be2d-03e5926d88bb None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:42.068262 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f91b6-0ff4-452e-8a9c-4125ae73ecfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:42.068853 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f91b6-0ff4-452e-8a9c-4125ae73ecfd admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:42.069167 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f91b6-0ff4-452e-8a9c-4125ae73ecfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:42.069443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f91b6-0ff4-452e-8a9c-4125ae73ecfd admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:42.070079 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f91b6-0ff4-452e-8a9c-4125ae73ecfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:42.071990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f91b6-0ff4-452e-8a9c-4125ae73ecfd admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:42.099469 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1422/2844] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:40:42 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:42.103768 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc517b-8e38-4544-998c-7111f199f3cc admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:42.114260 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc517b-8e38-4544-998c-7111f199f3cc admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:42.169780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e2881-05d7-4a21-8a28-1c33949bec82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:42.170389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e2881-05d7-4a21-8a28-1c33949bec82 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:42.170673 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e2881-05d7-4a21-8a28-1c33949bec82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:42.170951 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e2881-05d7-4a21-8a28-1c33949bec82 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:42.171661 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e2881-05d7-4a21-8a28-1c33949bec82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:42.173577 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e2881-05d7-4a21-8a28-1c33949bec82 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:42.194242 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1423/2845] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:40:42 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:42.198588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f91b6-0ff4-452e-8a9c-4125ae73ecfd admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:42.209711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f91b6-0ff4-452e-8a9c-4125ae73ecfd admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:42.214976 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2343ace-137e-4705-ad52-f63e96703fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:42.215605 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2343ace-137e-4705-ad52-f63e96703fca admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:42.215891 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2343ace-137e-4705-ad52-f63e96703fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:42.216163 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2343ace-137e-4705-ad52-f63e96703fca admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:42.216808 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2343ace-137e-4705-ad52-f63e96703fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:42.218400 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2343ace-137e-4705-ad52-f63e96703fca admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:42.228125 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1423/2846] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:42.232065 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e2881-05d7-4a21-8a28-1c33949bec82 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:42.241964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e2881-05d7-4a21-8a28-1c33949bec82 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:42.246875 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896bfa0-f9fb-4beb-91d7-51687a85d308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:42.247472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896bfa0-f9fb-4beb-91d7-51687a85d308 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:42.247792 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896bfa0-f9fb-4beb-91d7-51687a85d308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:42.248066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896bfa0-f9fb-4beb-91d7-51687a85d308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:42.248705 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896bfa0-f9fb-4beb-91d7-51687a85d308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:42.250350 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896bfa0-f9fb-4beb-91d7-51687a85d308 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:42.261814 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1424/2847] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:42.266040 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2343ace-137e-4705-ad52-f63e96703fca admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:42.276027 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2343ace-137e-4705-ad52-f63e96703fca admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:42.281342 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49f2f1-781f-456b-b79b-d56a64b82ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:42.281984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49f2f1-781f-456b-b79b-d56a64b82ff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:42.282269 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49f2f1-781f-456b-b79b-d56a64b82ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:42.282576 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49f2f1-781f-456b-b79b-d56a64b82ff6 admin admin] PATH_INFO: `/v3/projects/446108675fc749adb583f3e17024f3d9/users/90451d1815104689b58a61a17ae842b3/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:42.291026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:42.299071 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49f2f1-781f-456b-b79b-d56a64b82ff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=446108675fc749adb583f3e17024f3d9, user_id=90451d1815104689b58a61a17ae842b3, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:42.302330 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49f2f1-781f-456b-b79b-d56a64b82ff6 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:42.314912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1424/2848] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:40:42 2022] PUT /identity/v3/projects/446108675fc749adb583f3e17024f3d9/users/90451d1815104689b58a61a17ae842b3/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:42.326737 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadb377-c5bb-42c7-a06e-2ebac488f29c None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:42.327019 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadb377-c5bb-42c7-a06e-2ebac488f29c None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:42.327291 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadb377-c5bb-42c7-a06e-2ebac488f29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:42.355369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-bdadb377-c5bb-42c7-a06e-2ebac488f29c None None] MFA Rules not processed for user `90451d1815104689b58a61a17ae842b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:42.356618 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-bdadb377-c5bb-42c7-a06e-2ebac488f29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:42.419535 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1425/2849] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:42.424014 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49f2f1-781f-456b-b79b-d56a64b82ff6 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:42.484391 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49f2f1-781f-456b-b79b-d56a64b82ff6 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:42.488893 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3efca-195e-4c7a-98ca-35f187facb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:42.489488 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3efca-195e-4c7a-98ca-35f187facb32 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:42.489880 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3efca-195e-4c7a-98ca-35f187facb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:42.490158 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3efca-195e-4c7a-98ca-35f187facb32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:42.490782 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3efca-195e-4c7a-98ca-35f187facb32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:42.492691 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3efca-195e-4c7a-98ca-35f187facb32 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:42.519080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1425/2850] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:40:42 2022] POST /identity/v3/projects => generated 391 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:40:42.523032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadb377-c5bb-42c7-a06e-2ebac488f29c None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:42.534251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadb377-c5bb-42c7-a06e-2ebac488f29c None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:42.539031 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:42.591135 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175f951-4818-4861-9009-c136c1dbf2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:42.591763 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175f951-4818-4861-9009-c136c1dbf2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:42.592046 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175f951-4818-4861-9009-c136c1dbf2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:42.592319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175f951-4818-4861-9009-c136c1dbf2f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:42.592978 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175f951-4818-4861-9009-c136c1dbf2f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:42.594879 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175f951-4818-4861-9009-c136c1dbf2f6 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:42.615066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1426/2851] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:40:42 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:42.618937 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3efca-195e-4c7a-98ca-35f187facb32 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:42.629053 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3efca-195e-4c7a-98ca-35f187facb32 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:42.633657 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53495f6b-c302-4552-9d3a-d7e13106bad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:42.634250 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53495f6b-c302-4552-9d3a-d7e13106bad4 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:42.634529 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53495f6b-c302-4552-9d3a-d7e13106bad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:42.634805 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53495f6b-c302-4552-9d3a-d7e13106bad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:42.635441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53495f6b-c302-4552-9d3a-d7e13106bad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:42.637069 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53495f6b-c302-4552-9d3a-d7e13106bad4 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:42.647318 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1426/2852] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:42.651053 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175f951-4818-4861-9009-c136c1dbf2f6 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:42.661421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175f951-4818-4861-9009-c136c1dbf2f6 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:42.665982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1568b10-6880-4f06-ac4b-2525933be9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:42.666610 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1568b10-6880-4f06-ac4b-2525933be9b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:42.666891 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1568b10-6880-4f06-ac4b-2525933be9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:42.667166 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1568b10-6880-4f06-ac4b-2525933be9b9 admin admin] PATH_INFO: `/v3/projects/3e0960e8950241c9af23be8ca8550d68/users/5d60cbb28d9d40fc88a805e07014aaa3/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:42.682251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1568b10-6880-4f06-ac4b-2525933be9b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e0960e8950241c9af23be8ca8550d68, user_id=5d60cbb28d9d40fc88a805e07014aaa3, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:42.684975 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1568b10-6880-4f06-ac4b-2525933be9b9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:42.698067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1427/2853] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:40:42 2022] PUT /identity/v3/projects/3e0960e8950241c9af23be8ca8550d68/users/5d60cbb28d9d40fc88a805e07014aaa3/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:42.702064 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53495f6b-c302-4552-9d3a-d7e13106bad4 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:42.711847 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53495f6b-c302-4552-9d3a-d7e13106bad4 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:42.769048 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac32b2-a1aa-40c0-b4fc-2900b23b3f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:42.769655 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac32b2-a1aa-40c0-b4fc-2900b23b3f29 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:42.769935 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac32b2-a1aa-40c0-b4fc-2900b23b3f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:42.770206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac32b2-a1aa-40c0-b4fc-2900b23b3f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:42.770871 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac32b2-a1aa-40c0-b4fc-2900b23b3f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:42.772497 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac32b2-a1aa-40c0-b4fc-2900b23b3f29 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:42.782842 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1427/2854] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:42.787098 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1568b10-6880-4f06-ac4b-2525933be9b9 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:42.797908 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1568b10-6880-4f06-ac4b-2525933be9b9 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:42.802544 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09669c32-a5a1-4006-8cc7-7cad649022e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:42.803190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09669c32-a5a1-4006-8cc7-7cad649022e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:42.803477 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09669c32-a5a1-4006-8cc7-7cad649022e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:42.803786 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09669c32-a5a1-4006-8cc7-7cad649022e2 admin admin] PATH_INFO: `/v3/projects/3e0960e8950241c9af23be8ca8550d68/users/5d60cbb28d9d40fc88a805e07014aaa3/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:42.805688 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09669c32-a5a1-4006-8cc7-7cad649022e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e0960e8950241c9af23be8ca8550d68, user_id=5d60cbb28d9d40fc88a805e07014aaa3, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:42.808341 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09669c32-a5a1-4006-8cc7-7cad649022e2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:42.818633 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1428/2855] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:40:42 2022] PUT /identity/v3/projects/3e0960e8950241c9af23be8ca8550d68/users/5d60cbb28d9d40fc88a805e07014aaa3/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:42.828861 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061ca18-f585-4948-9e12-1642c4a5128e None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:42.829145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061ca18-f585-4948-9e12-1642c4a5128e None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:42.829418 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061ca18-f585-4948-9e12-1642c4a5128e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:42.859118 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3061ca18-f585-4948-9e12-1642c4a5128e None None] MFA Rules not processed for user `5d60cbb28d9d40fc88a805e07014aaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:42.860378 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3061ca18-f585-4948-9e12-1642c4a5128e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:42.903569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:42.928206 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1428/2856] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:42 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:40:42.939583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09669c32-a5a1-4006-8cc7-7cad649022e2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:42.941433 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-09669c32-a5a1-4006-8cc7-7cad649022e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:42.997408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09669c32-a5a1-4006-8cc7-7cad649022e2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:43.001010 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45063b40-d883-44fe-8aa6-0cdec02c43af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:43.001601 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45063b40-d883-44fe-8aa6-0cdec02c43af service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:43.001877 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45063b40-d883-44fe-8aa6-0cdec02c43af service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:43.002145 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45063b40-d883-44fe-8aa6-0cdec02c43af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:43.010925 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45063b40-d883-44fe-8aa6-0cdec02c43af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:43.012439 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45063b40-d883-44fe-8aa6-0cdec02c43af service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:43.021689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1429/2857] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:40:42 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:40:43.068298 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061ca18-f585-4948-9e12-1642c4a5128e None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:43.069709 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3061ca18-f585-4948-9e12-1642c4a5128e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:43.081638 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061ca18-f585-4948-9e12-1642c4a5128e None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:43.085221 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ab412-8778-4128-9ae5-8684ad6cead5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:43.085815 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ab412-8778-4128-9ae5-8684ad6cead5 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:43.086096 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ab412-8778-4128-9ae5-8684ad6cead5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:43.086378 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ab412-8778-4128-9ae5-8684ad6cead5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:43.141810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ab412-8778-4128-9ae5-8684ad6cead5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:43.143242 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ab412-8778-4128-9ae5-8684ad6cead5 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:43.152090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1429/2858] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:40:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:40:46.393848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45063b40-d883-44fe-8aa6-0cdec02c43af service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:46.454814 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45063b40-d883-44fe-8aa6-0cdec02c43af service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:46.459313 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a703f-6ae1-4c42-ab1a-545751c44897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:46.459964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a703f-6ae1-4c42-ab1a-545751c44897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:46.460251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a703f-6ae1-4c42-ab1a-545751c44897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:46.460531 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a703f-6ae1-4c42-ab1a-545751c44897 admin admin] PATH_INFO: `/v3/users/90451d1815104689b58a61a17ae842b3` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:46.461637 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a703f-6ae1-4c42-ab1a-545751c44897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90451d1815104689b58a61a17ae842b3)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:46.463575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a703f-6ae1-4c42-ab1a-545751c44897 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:46.468572 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:46.510799 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d60a703f-6ae1-4c42-ab1a-545751c44897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90451d1815104689b58a61a17ae842b3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:46.517969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d60a703f-6ae1-4c42-ab1a-545751c44897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90451d1815104689b58a61a17ae842b3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:46.518667 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d60a703f-6ae1-4c42-ab1a-545751c44897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90451d1815104689b58a61a17ae842b3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:46.519298 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d60a703f-6ae1-4c42-ab1a-545751c44897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90451d1815104689b58a61a17ae842b3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:46.525926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-d60a703f-6ae1-4c42-ab1a-545751c44897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90451d1815104689b58a61a17ae842b3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:46.544307 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1430/2859] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:40:46 2022] DELETE /identity/v3/users/90451d1815104689b58a61a17ae842b3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:46.548765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620a8bc-87ac-4138-b894-b1c6352a30b8 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:46.549283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620a8bc-87ac-4138-b894-b1c6352a30b8 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:46.549283 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620a8bc-87ac-4138-b894-b1c6352a30b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:46.567864 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-3620a8bc-87ac-4138-b894-b1c6352a30b8 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:46.569042 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3620a8bc-87ac-4138-b894-b1c6352a30b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:46.625376 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1430/2860] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:46.634947 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a703f-6ae1-4c42-ab1a-545751c44897 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:46.636929 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-d60a703f-6ae1-4c42-ab1a-545751c44897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:46.691961 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a703f-6ae1-4c42-ab1a-545751c44897 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:46.695548 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01862089-f951-46ae-b108-c8110ae8fb39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:46.696141 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01862089-f951-46ae-b108-c8110ae8fb39 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:46.696420 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01862089-f951-46ae-b108-c8110ae8fb39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:46.696687 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01862089-f951-46ae-b108-c8110ae8fb39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:46.705861 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01862089-f951-46ae-b108-c8110ae8fb39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:46.707273 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01862089-f951-46ae-b108-c8110ae8fb39 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:46.717146 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1431/2861] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:40:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:40:46.981426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620a8bc-87ac-4138-b894-b1c6352a30b8 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:46.982928 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3620a8bc-87ac-4138-b894-b1c6352a30b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:46.995357 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620a8bc-87ac-4138-b894-b1c6352a30b8 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:47.000042 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc13736-dcc5-4eb2-9278-568bceda3c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:47.000766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc13736-dcc5-4eb2-9278-568bceda3c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:47.001112 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc13736-dcc5-4eb2-9278-568bceda3c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:47.001448 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc13736-dcc5-4eb2-9278-568bceda3c69 admin admin] PATH_INFO: `/v3/projects/446108675fc749adb583f3e17024f3d9` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:47.002644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc13736-dcc5-4eb2-9278-568bceda3c69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=446108675fc749adb583f3e17024f3d9)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:47.004646 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc13736-dcc5-4eb2-9278-568bceda3c69 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:47.052036 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fbc13736-dcc5-4eb2-9278-568bceda3c69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '446108675fc749adb583f3e17024f3d9', 'request_id': 'req-fbc13736-dcc5-4eb2-9278-568bceda3c69'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:47.059026 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fbc13736-dcc5-4eb2-9278-568bceda3c69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '446108675fc749adb583f3e17024f3d9', 'request_id': 'req-fbc13736-dcc5-4eb2-9278-568bceda3c69'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:47.065106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fbc13736-dcc5-4eb2-9278-568bceda3c69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '446108675fc749adb583f3e17024f3d9', 'request_id': 'req-fbc13736-dcc5-4eb2-9278-568bceda3c69'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:47.075224 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fbc13736-dcc5-4eb2-9278-568bceda3c69 admin admin] The token cache is being invalidate because project 446108675fc749adb583f3e17024f3d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:40:47.075524 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fbc13736-dcc5-4eb2-9278-568bceda3c69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:47.076182 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-fbc13736-dcc5-4eb2-9278-568bceda3c69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:47.077644 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1431/2862] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:40:46 2022] DELETE /identity/v3/projects/446108675fc749adb583f3e17024f3d9 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:47.081615 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01862089-f951-46ae-b108-c8110ae8fb39 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:47.092120 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01862089-f951-46ae-b108-c8110ae8fb39 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:47.147205 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36387f47-64cd-41d9-ab39-c02b0fa06511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:47.147865 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36387f47-64cd-41d9-ab39-c02b0fa06511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:47.148150 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36387f47-64cd-41d9-ab39-c02b0fa06511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:47.148425 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36387f47-64cd-41d9-ab39-c02b0fa06511 admin admin] PATH_INFO: `/v3/users/5d60cbb28d9d40fc88a805e07014aaa3` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:47.149637 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36387f47-64cd-41d9-ab39-c02b0fa06511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d60cbb28d9d40fc88a805e07014aaa3)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:47.152108 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36387f47-64cd-41d9-ab39-c02b0fa06511 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:47.180602 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:47.200824 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-36387f47-64cd-41d9-ab39-c02b0fa06511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d60cbb28d9d40fc88a805e07014aaa3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:47.207583 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-36387f47-64cd-41d9-ab39-c02b0fa06511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d60cbb28d9d40fc88a805e07014aaa3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:47.208241 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-36387f47-64cd-41d9-ab39-c02b0fa06511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d60cbb28d9d40fc88a805e07014aaa3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:47.208851 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-36387f47-64cd-41d9-ab39-c02b0fa06511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d60cbb28d9d40fc88a805e07014aaa3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:47.216268 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-36387f47-64cd-41d9-ab39-c02b0fa06511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d60cbb28d9d40fc88a805e07014aaa3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:47.233946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1432/2863] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:40:47 2022] DELETE /identity/v3/users/5d60cbb28d9d40fc88a805e07014aaa3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:47.499186 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc13736-dcc5-4eb2-9278-568bceda3c69 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:47.500965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-fbc13736-dcc5-4eb2-9278-568bceda3c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:47.512703 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:47.515467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc13736-dcc5-4eb2-9278-568bceda3c69 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:47.575003 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e40cd-8a63-43dc-8d70-57f007dc5d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:47.575728 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e40cd-8a63-43dc-8d70-57f007dc5d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:47.575912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e40cd-8a63-43dc-8d70-57f007dc5d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:47.576180 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e40cd-8a63-43dc-8d70-57f007dc5d2e admin admin] PATH_INFO: `/v3/projects/3e0960e8950241c9af23be8ca8550d68` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:47.577270 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e40cd-8a63-43dc-8d70-57f007dc5d2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e0960e8950241c9af23be8ca8550d68)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:47.579144 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e40cd-8a63-43dc-8d70-57f007dc5d2e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:47.624004 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a00e40cd-8a63-43dc-8d70-57f007dc5d2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e0960e8950241c9af23be8ca8550d68', 'request_id': 'req-a00e40cd-8a63-43dc-8d70-57f007dc5d2e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:47.631263 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a00e40cd-8a63-43dc-8d70-57f007dc5d2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e0960e8950241c9af23be8ca8550d68', 'request_id': 'req-a00e40cd-8a63-43dc-8d70-57f007dc5d2e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:47.637360 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a00e40cd-8a63-43dc-8d70-57f007dc5d2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e0960e8950241c9af23be8ca8550d68', 'request_id': 'req-a00e40cd-8a63-43dc-8d70-57f007dc5d2e'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:47.647352 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a00e40cd-8a63-43dc-8d70-57f007dc5d2e admin admin] The token cache is being invalidate because project 3e0960e8950241c9af23be8ca8550d68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:40:47.647731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a00e40cd-8a63-43dc-8d70-57f007dc5d2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:47.648352 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-a00e40cd-8a63-43dc-8d70-57f007dc5d2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:47.649793 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1432/2864] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:40:47 2022] DELETE /identity/v3/projects/3e0960e8950241c9af23be8ca8550d68 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:47.654625 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22fa8d-22cc-43b2-9ab1-a02c40166545 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:47.655108 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22fa8d-22cc-43b2-9ab1-a02c40166545 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:47.655210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22fa8d-22cc-43b2-9ab1-a02c40166545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:47.674967 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7f22fa8d-22cc-43b2-9ab1-a02c40166545 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:47.676179 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7f22fa8d-22cc-43b2-9ab1-a02c40166545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:47.733162 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1433/2865] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:47.740859 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489bc35-1cdb-4a4f-b77b-6acc698d93a1 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:47.741147 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489bc35-1cdb-4a4f-b77b-6acc698d93a1 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:47.741469 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489bc35-1cdb-4a4f-b77b-6acc698d93a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:47.761527 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-f489bc35-1cdb-4a4f-b77b-6acc698d93a1 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:47.762720 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-f489bc35-1cdb-4a4f-b77b-6acc698d93a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:47.778161 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1433/2866] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:47.782689 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22fa8d-22cc-43b2-9ab1-a02c40166545 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:47.793379 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22fa8d-22cc-43b2-9ab1-a02c40166545 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:47.797943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6632c-7deb-4c62-8803-da65d48c81e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:47.798542 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6632c-7deb-4c62-8803-da65d48c81e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:47.798823 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6632c-7deb-4c62-8803-da65d48c81e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:47.799095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6632c-7deb-4c62-8803-da65d48c81e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:47.799772 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6632c-7deb-4c62-8803-da65d48c81e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:40:47.800077 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6632c-7deb-4c62-8803-da65d48c81e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:47.801666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6632c-7deb-4c62-8803-da65d48c81e4 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:47.812969 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1434/2867] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:40:47.816981 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489bc35-1cdb-4a4f-b77b-6acc698d93a1 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:47.826047 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489bc35-1cdb-4a4f-b77b-6acc698d93a1 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:47.830626 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498e2df-dce3-44b1-84db-1f6c72b7da51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:47.831216 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498e2df-dce3-44b1-84db-1f6c72b7da51 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:47.831496 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498e2df-dce3-44b1-84db-1f6c72b7da51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:47.831803 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498e2df-dce3-44b1-84db-1f6c72b7da51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:47.832475 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498e2df-dce3-44b1-84db-1f6c72b7da51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:47.834331 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498e2df-dce3-44b1-84db-1f6c72b7da51 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:47.862151 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1434/2868] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:40:47 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:47.866172 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6632c-7deb-4c62-8803-da65d48c81e4 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:47.876001 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6632c-7deb-4c62-8803-da65d48c81e4 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:47.930671 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6809-0668-4972-8960-9e3eea629230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:47.931271 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6809-0668-4972-8960-9e3eea629230 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:47.931581 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6809-0668-4972-8960-9e3eea629230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:47.931856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6809-0668-4972-8960-9e3eea629230 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:47.932499 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a6809-0668-4972-8960-9e3eea629230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:47.934434 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a6809-0668-4972-8960-9e3eea629230 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:47.954971 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1435/2869] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:40:47 2022] POST /identity/v3/users => generated 283 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:47.958987 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498e2df-dce3-44b1-84db-1f6c72b7da51 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:47.969467 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498e2df-dce3-44b1-84db-1f6c72b7da51 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:47.973973 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227961b-c0b5-463f-ac40-5ab928a57144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:47.974560 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227961b-c0b5-463f-ac40-5ab928a57144 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:47.974837 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227961b-c0b5-463f-ac40-5ab928a57144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:47.975105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227961b-c0b5-463f-ac40-5ab928a57144 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:47.975766 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227961b-c0b5-463f-ac40-5ab928a57144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:47.977344 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227961b-c0b5-463f-ac40-5ab928a57144 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:47.987477 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1435/2870] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:47.991076 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6809-0668-4972-8960-9e3eea629230 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:48.000943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6809-0668-4972-8960-9e3eea629230 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:48.005530 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6df294-806c-490a-be46-c384fd682a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:48.006109 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6df294-806c-490a-be46-c384fd682a8b admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:48.006389 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6df294-806c-490a-be46-c384fd682a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:48.006660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6df294-806c-490a-be46-c384fd682a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:48.007291 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6df294-806c-490a-be46-c384fd682a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:48.008907 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6df294-806c-490a-be46-c384fd682a8b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:48.019294 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1436/2871] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:48.023104 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227961b-c0b5-463f-ac40-5ab928a57144 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:48.033709 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227961b-c0b5-463f-ac40-5ab928a57144 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:48.038558 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0ab85-87c6-49a4-97a1-a22e2d51c936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:48.039190 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0ab85-87c6-49a4-97a1-a22e2d51c936 admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:48.039478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0ab85-87c6-49a4-97a1-a22e2d51c936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:48.039783 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0ab85-87c6-49a4-97a1-a22e2d51c936 admin admin] PATH_INFO: `/v3/projects/d42b93b79b4a4db99bcaf3640448495e/users/e200789138f14a999437c7af94fcc0ce/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:48.054510 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0ab85-87c6-49a4-97a1-a22e2d51c936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d42b93b79b4a4db99bcaf3640448495e, user_id=e200789138f14a999437c7af94fcc0ce, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:48.057167 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0ab85-87c6-49a4-97a1-a22e2d51c936 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:48.069863 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1436/2872] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:40:48 2022] PUT /identity/v3/projects/d42b93b79b4a4db99bcaf3640448495e/users/e200789138f14a999437c7af94fcc0ce/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 12:40:48.081194 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1091f-e38e-42fc-a7bf-3a6143c40a54 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:48.081487 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1091f-e38e-42fc-a7bf-3a6143c40a54 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:48.081785 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1091f-e38e-42fc-a7bf-3a6143c40a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:48.102379 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:48.111595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-65c1091f-e38e-42fc-a7bf-3a6143c40a54 None None] MFA Rules not processed for user `e200789138f14a999437c7af94fcc0ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:48.113507 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-65c1091f-e38e-42fc-a7bf-3a6143c40a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:48.179891 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1437/2873] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:48.184903 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0ab85-87c6-49a4-97a1-a22e2d51c936 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:48.216999 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:48.247674 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0ab85-87c6-49a4-97a1-a22e2d51c936 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:48.252301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678bca1-09e8-481b-86bd-7d1ec9d84a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:48.252912 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678bca1-09e8-481b-86bd-7d1ec9d84a1e admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:48.253191 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678bca1-09e8-481b-86bd-7d1ec9d84a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:48.253463 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678bca1-09e8-481b-86bd-7d1ec9d84a1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:48.254103 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e678bca1-09e8-481b-86bd-7d1ec9d84a1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:48.256051 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e678bca1-09e8-481b-86bd-7d1ec9d84a1e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:48.283225 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1437/2874] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:40:48 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:48.287642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1091f-e38e-42fc-a7bf-3a6143c40a54 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:48.298562 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1091f-e38e-42fc-a7bf-3a6143c40a54 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:48.353334 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76add64c-486f-4236-8ba5-c448e67b2468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:48.353934 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76add64c-486f-4236-8ba5-c448e67b2468 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:48.354232 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76add64c-486f-4236-8ba5-c448e67b2468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:48.354500 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76add64c-486f-4236-8ba5-c448e67b2468 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:48.355684 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76add64c-486f-4236-8ba5-c448e67b2468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:48.357602 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76add64c-486f-4236-8ba5-c448e67b2468 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:48.378459 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1438/2875] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:40:48 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:48.382416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678bca1-09e8-481b-86bd-7d1ec9d84a1e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:48.392969 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678bca1-09e8-481b-86bd-7d1ec9d84a1e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:48.397582 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ffa31-7a49-4356-98d3-69c6a14a0ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:48.398175 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ffa31-7a49-4356-98d3-69c6a14a0ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:48.398454 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ffa31-7a49-4356-98d3-69c6a14a0ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:48.398724 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ffa31-7a49-4356-98d3-69c6a14a0ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:48.399356 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ffa31-7a49-4356-98d3-69c6a14a0ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:48.400957 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ffa31-7a49-4356-98d3-69c6a14a0ca0 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:48.410545 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1438/2876] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:48.414858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76add64c-486f-4236-8ba5-c448e67b2468 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:48.425227 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76add64c-486f-4236-8ba5-c448e67b2468 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:48.429834 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc2150-d5c7-4cda-84ed-cf806f135a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:48.430517 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc2150-d5c7-4cda-84ed-cf806f135a6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:48.430750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc2150-d5c7-4cda-84ed-cf806f135a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:48.431023 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc2150-d5c7-4cda-84ed-cf806f135a6e admin admin] PATH_INFO: `/v3/projects/bd867917a6194dccbd70a43e86d00a97/users/562d46cbc0604297a18b33c62e28d8de/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:48.446736 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc2150-d5c7-4cda-84ed-cf806f135a6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd867917a6194dccbd70a43e86d00a97, user_id=562d46cbc0604297a18b33c62e28d8de, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:48.449465 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc2150-d5c7-4cda-84ed-cf806f135a6e admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:48.462139 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1439/2877] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:40:48 2022] PUT /identity/v3/projects/bd867917a6194dccbd70a43e86d00a97/users/562d46cbc0604297a18b33c62e28d8de/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:48.466179 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ffa31-7a49-4356-98d3-69c6a14a0ca0 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:48.476148 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ffa31-7a49-4356-98d3-69c6a14a0ca0 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:48.531692 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a475c84-1ae0-45c5-8c41-8476fb6c145c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:48.532314 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a475c84-1ae0-45c5-8c41-8476fb6c145c admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:48.532581 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a475c84-1ae0-45c5-8c41-8476fb6c145c admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:48.532857 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a475c84-1ae0-45c5-8c41-8476fb6c145c admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:48.533501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a475c84-1ae0-45c5-8c41-8476fb6c145c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:48.535083 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a475c84-1ae0-45c5-8c41-8476fb6c145c admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:48.545443 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1439/2878] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:48.549501 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc2150-d5c7-4cda-84ed-cf806f135a6e admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:48.559851 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc2150-d5c7-4cda-84ed-cf806f135a6e admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:48.564490 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72b21a-916d-4197-a06b-2162fe732ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:48.565132 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72b21a-916d-4197-a06b-2162fe732ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:48.565414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72b21a-916d-4197-a06b-2162fe732ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:48.565688 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72b21a-916d-4197-a06b-2162fe732ea9 admin admin] PATH_INFO: `/v3/projects/bd867917a6194dccbd70a43e86d00a97/users/562d46cbc0604297a18b33c62e28d8de/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:48.567642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72b21a-916d-4197-a06b-2162fe732ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd867917a6194dccbd70a43e86d00a97, user_id=562d46cbc0604297a18b33c62e28d8de, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:48.570257 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72b21a-916d-4197-a06b-2162fe732ea9 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:48.580344 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1440/2879] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:40:48 2022] PUT /identity/v3/projects/bd867917a6194dccbd70a43e86d00a97/users/562d46cbc0604297a18b33c62e28d8de/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:48.590706 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa9d98-f8f0-4d64-b597-90eb4e35d7f7 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:48.591198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa9d98-f8f0-4d64-b597-90eb4e35d7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:48.591304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa9d98-f8f0-4d64-b597-90eb4e35d7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:48.618805 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-67fa9d98-f8f0-4d64-b597-90eb4e35d7f7 None None] MFA Rules not processed for user `562d46cbc0604297a18b33c62e28d8de`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:48.620041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-67fa9d98-f8f0-4d64-b597-90eb4e35d7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:48.686961 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1440/2880] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 12:40:48.698043 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72b21a-916d-4197-a06b-2162fe732ea9 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:48.699783 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2a72b21a-916d-4197-a06b-2162fe732ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:48.757319 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72b21a-916d-4197-a06b-2162fe732ea9 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:48.760980 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a857d59-1393-40c5-ba44-5050b37ce517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:48.761578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a857d59-1393-40c5-ba44-5050b37ce517 service cinder] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:48.761858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a857d59-1393-40c5-ba44-5050b37ce517 service cinder] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:48.762130 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a857d59-1393-40c5-ba44-5050b37ce517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:48.769398 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:48.771919 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a857d59-1393-40c5-ba44-5050b37ce517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:48.773837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a857d59-1393-40c5-ba44-5050b37ce517 service cinder] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:48.785481 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1441/2881] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:40:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:40:48.868568 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa9d98-f8f0-4d64-b597-90eb4e35d7f7 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:48.878146 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa9d98-f8f0-4d64-b597-90eb4e35d7f7 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:48.881735 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e75c15-30a3-487e-b2a5-2cde016d0de8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:48.882325 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e75c15-30a3-487e-b2a5-2cde016d0de8 service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:48.882599 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e75c15-30a3-487e-b2a5-2cde016d0de8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:48.882865 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e75c15-30a3-487e-b2a5-2cde016d0de8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:48.907464 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:48.940588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e75c15-30a3-487e-b2a5-2cde016d0de8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:48.941944 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e75c15-30a3-487e-b2a5-2cde016d0de8 service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:48.950607 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1441/2882] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:40:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:40:53.618964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a857d59-1393-40c5-ba44-5050b37ce517 service cinder] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:53.683538 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a857d59-1393-40c5-ba44-5050b37ce517 service cinder] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:53.688089 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e21735-6c1e-4444-b9c9-fa9a9ad9a69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:53.688713 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e21735-6c1e-4444-b9c9-fa9a9ad9a69f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:53.688996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e21735-6c1e-4444-b9c9-fa9a9ad9a69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:53.689269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e21735-6c1e-4444-b9c9-fa9a9ad9a69f admin admin] PATH_INFO: `/v3/users/e200789138f14a999437c7af94fcc0ce` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:53.690414 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e21735-6c1e-4444-b9c9-fa9a9ad9a69f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e200789138f14a999437c7af94fcc0ce)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:53.692355 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e21735-6c1e-4444-b9c9-fa9a9ad9a69f admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:53.737578 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-62e21735-6c1e-4444-b9c9-fa9a9ad9a69f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e200789138f14a999437c7af94fcc0ce'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:53.744411 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-62e21735-6c1e-4444-b9c9-fa9a9ad9a69f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e200789138f14a999437c7af94fcc0ce'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:53.745067 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-62e21735-6c1e-4444-b9c9-fa9a9ad9a69f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e200789138f14a999437c7af94fcc0ce'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:53.746214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-62e21735-6c1e-4444-b9c9-fa9a9ad9a69f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e200789138f14a999437c7af94fcc0ce'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:53.753507 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-62e21735-6c1e-4444-b9c9-fa9a9ad9a69f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e200789138f14a999437c7af94fcc0ce'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:53.770694 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1442/2883] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:40:53 2022] DELETE /identity/v3/users/e200789138f14a999437c7af94fcc0ce => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:53.775145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b659d-0a95-47dc-b816-150bc105a034 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:53.775682 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b659d-0a95-47dc-b816-150bc105a034 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:53.775804 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b659d-0a95-47dc-b816-150bc105a034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:53.796319 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-848b659d-0a95-47dc-b816-150bc105a034 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:53.797767 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-848b659d-0a95-47dc-b816-150bc105a034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:53.857093 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1442/2884] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:53.866856 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e21735-6c1e-4444-b9c9-fa9a9ad9a69f admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:53.868285 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-62e21735-6c1e-4444-b9c9-fa9a9ad9a69f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:53.924906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e21735-6c1e-4444-b9c9-fa9a9ad9a69f admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:53.928562 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea72018-3834-4881-af96-553907bbe756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:53.929160 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea72018-3834-4881-af96-553907bbe756 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:53.929441 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea72018-3834-4881-af96-553907bbe756 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:53.929714 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea72018-3834-4881-af96-553907bbe756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:53.938524 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea72018-3834-4881-af96-553907bbe756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:53.939998 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea72018-3834-4881-af96-553907bbe756 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:53.949230 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1443/2885] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:40:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:40:54.211229 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b659d-0a95-47dc-b816-150bc105a034 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:54.212731 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-848b659d-0a95-47dc-b816-150bc105a034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:54.225042 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b659d-0a95-47dc-b816-150bc105a034 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:54.230614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:54.231257 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:54.231603 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:54.231880 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38 admin admin] PATH_INFO: `/v3/projects/d42b93b79b4a4db99bcaf3640448495e` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:54.233016 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d42b93b79b4a4db99bcaf3640448495e)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:54.234921 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:54.282263 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd42b93b79b4a4db99bcaf3640448495e', 'request_id': 'req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:54.289583 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd42b93b79b4a4db99bcaf3640448495e', 'request_id': 'req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:54.296223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd42b93b79b4a4db99bcaf3640448495e', 'request_id': 'req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:54.306438 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38 admin admin] The token cache is being invalidate because project d42b93b79b4a4db99bcaf3640448495e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:40:54.306727 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:54.307357 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:54.309517 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1443/2886] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:40:54 2022] DELETE /identity/v3/projects/d42b93b79b4a4db99bcaf3640448495e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:54.313619 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea72018-3834-4881-af96-553907bbe756 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:54.323383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea72018-3834-4881-af96-553907bbe756 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:54.335725 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:54.380592 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b5d0d-bdc5-4e34-a4fc-7730ed9631e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:54.381312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b5d0d-bdc5-4e34-a4fc-7730ed9631e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:54.381591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b5d0d-bdc5-4e34-a4fc-7730ed9631e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:54.381888 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b5d0d-bdc5-4e34-a4fc-7730ed9631e7 admin admin] PATH_INFO: `/v3/users/562d46cbc0604297a18b33c62e28d8de` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:54.382990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b5d0d-bdc5-4e34-a4fc-7730ed9631e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=562d46cbc0604297a18b33c62e28d8de)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:54.384921 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b5d0d-bdc5-4e34-a4fc-7730ed9631e7 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:54.429858 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-610b5d0d-bdc5-4e34-a4fc-7730ed9631e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '562d46cbc0604297a18b33c62e28d8de'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:54.438046 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-610b5d0d-bdc5-4e34-a4fc-7730ed9631e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '562d46cbc0604297a18b33c62e28d8de'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:54.438695 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-610b5d0d-bdc5-4e34-a4fc-7730ed9631e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '562d46cbc0604297a18b33c62e28d8de'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:54.439295 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-610b5d0d-bdc5-4e34-a4fc-7730ed9631e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '562d46cbc0604297a18b33c62e28d8de'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:54.446068 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-610b5d0d-bdc5-4e34-a4fc-7730ed9631e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '562d46cbc0604297a18b33c62e28d8de'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:54.463044 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1444/2887] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:40:54 2022] DELETE /identity/v3/users/562d46cbc0604297a18b33c62e28d8de => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:54.723704 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:54.725250 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:54.739105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ae9e9-fb1c-4663-9e57-e6a7ea714f38 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:54.782364 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:54.796039 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44a628-1232-4b79-ba61-67b306afb404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:54.796658 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44a628-1232-4b79-ba61-67b306afb404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:54.796936 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44a628-1232-4b79-ba61-67b306afb404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:54.797209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44a628-1232-4b79-ba61-67b306afb404 admin admin] PATH_INFO: `/v3/projects/bd867917a6194dccbd70a43e86d00a97` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:54.798317 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44a628-1232-4b79-ba61-67b306afb404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd867917a6194dccbd70a43e86d00a97)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:54.800234 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44a628-1232-4b79-ba61-67b306afb404 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:54.845174 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ed44a628-1232-4b79-ba61-67b306afb404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd867917a6194dccbd70a43e86d00a97', 'request_id': 'req-ed44a628-1232-4b79-ba61-67b306afb404'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:54.851816 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ed44a628-1232-4b79-ba61-67b306afb404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd867917a6194dccbd70a43e86d00a97', 'request_id': 'req-ed44a628-1232-4b79-ba61-67b306afb404'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:54.858542 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ed44a628-1232-4b79-ba61-67b306afb404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd867917a6194dccbd70a43e86d00a97', 'request_id': 'req-ed44a628-1232-4b79-ba61-67b306afb404'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:54.869157 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ed44a628-1232-4b79-ba61-67b306afb404 admin admin] The token cache is being invalidate because project bd867917a6194dccbd70a43e86d00a97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:40:54.869444 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ed44a628-1232-4b79-ba61-67b306afb404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:54.870106 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-ed44a628-1232-4b79-ba61-67b306afb404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:40:54.871580 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1444/2888] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:40:54 2022] DELETE /identity/v3/projects/bd867917a6194dccbd70a43e86d00a97 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:54.876375 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbac184-c5e1-4f8c-8169-2dcf6eb58a81 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:54.876845 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbac184-c5e1-4f8c-8169-2dcf6eb58a81 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:54.876973 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbac184-c5e1-4f8c-8169-2dcf6eb58a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:54.896751 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-8cbac184-c5e1-4f8c-8169-2dcf6eb58a81 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:54.897937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8cbac184-c5e1-4f8c-8169-2dcf6eb58a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:54.956250 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1445/2889] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:54.965159 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2437127-1194-4089-9b75-a41e230d5099 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:54.965441 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2437127-1194-4089-9b75-a41e230d5099 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:54.965711 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2437127-1194-4089-9b75-a41e230d5099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:54.985471 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-e2437127-1194-4089-9b75-a41e230d5099 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:54.986653 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e2437127-1194-4089-9b75-a41e230d5099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:54.994362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1445/2890] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:54.998990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbac184-c5e1-4f8c-8169-2dcf6eb58a81 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:55.008312 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbac184-c5e1-4f8c-8169-2dcf6eb58a81 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:55.013306 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a764a33-d41c-4b87-a375-f95c31a600ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:55.013894 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a764a33-d41c-4b87-a375-f95c31a600ce admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:55.014174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a764a33-d41c-4b87-a375-f95c31a600ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:55.014442 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a764a33-d41c-4b87-a375-f95c31a600ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:55.015072 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a764a33-d41c-4b87-a375-f95c31a600ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:40:55.015369 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a764a33-d41c-4b87-a375-f95c31a600ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:55.016990 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a764a33-d41c-4b87-a375-f95c31a600ce admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:55.028106 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1446/2891] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:40:55.032387 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2437127-1194-4089-9b75-a41e230d5099 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:55.041796 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2437127-1194-4089-9b75-a41e230d5099 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:55.046949 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e4734-0603-4d91-8b3f-fbc51917f96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:55.047565 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e4734-0603-4d91-8b3f-fbc51917f96a admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:55.047845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e4734-0603-4d91-8b3f-fbc51917f96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:55.048113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e4734-0603-4d91-8b3f-fbc51917f96a admin admin] PATH_INFO: `/v3/projects` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:55.048811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e4734-0603-4d91-8b3f-fbc51917f96a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:55.050687 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e4734-0603-4d91-8b3f-fbc51917f96a admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:55.077980 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1446/2892] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:40:55 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:40:55.082248 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a764a33-d41c-4b87-a375-f95c31a600ce admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:55.092131 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a764a33-d41c-4b87-a375-f95c31a600ce admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:55.112655 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:55.151277 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d88b8-54fc-4358-b6fd-5d910b228636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:55.151926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d88b8-54fc-4358-b6fd-5d910b228636 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:55.152205 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d88b8-54fc-4358-b6fd-5d910b228636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:55.152475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d88b8-54fc-4358-b6fd-5d910b228636 admin admin] PATH_INFO: `/v3/users` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:55.153093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d88b8-54fc-4358-b6fd-5d910b228636 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:55.155021 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d88b8-54fc-4358-b6fd-5d910b228636 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:55.176179 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1447/2893] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:40:55 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:40:55.180268 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e4734-0603-4d91-8b3f-fbc51917f96a admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:55.191165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e4734-0603-4d91-8b3f-fbc51917f96a admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:55.195826 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6faf4d-3e21-4050-8520-a3c5dc3b7880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:55.196422 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6faf4d-3e21-4050-8520-a3c5dc3b7880 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:55.196704 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6faf4d-3e21-4050-8520-a3c5dc3b7880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:55.196976 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6faf4d-3e21-4050-8520-a3c5dc3b7880 admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:55.197614 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6faf4d-3e21-4050-8520-a3c5dc3b7880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:55.199197 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6faf4d-3e21-4050-8520-a3c5dc3b7880 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:55.208946 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1447/2894] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:55.212702 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d88b8-54fc-4358-b6fd-5d910b228636 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:55.222625 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d88b8-54fc-4358-b6fd-5d910b228636 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:55.227421 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ff65c-33f1-4e8d-84d3-81ff8c2b1e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:55.228039 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ff65c-33f1-4e8d-84d3-81ff8c2b1e1d admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:55.228320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ff65c-33f1-4e8d-84d3-81ff8c2b1e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:55.228591 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ff65c-33f1-4e8d-84d3-81ff8c2b1e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:55.229226 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ff65c-33f1-4e8d-84d3-81ff8c2b1e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:55.231303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ff65c-33f1-4e8d-84d3-81ff8c2b1e1d admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:55.241066 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1448/2895] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:40:55.245777 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6faf4d-3e21-4050-8520-a3c5dc3b7880 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:55.255775 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6faf4d-3e21-4050-8520-a3c5dc3b7880 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:55.260408 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51023a-b844-483e-a108-bec0227fac2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:55.261055 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51023a-b844-483e-a108-bec0227fac2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:55.261365 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51023a-b844-483e-a108-bec0227fac2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:55.261640 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51023a-b844-483e-a108-bec0227fac2e admin admin] PATH_INFO: `/v3/projects/352ca3049c444444a8ce0c29170dd517/users/5c9dbce0a6f9482397a79cd734ebab07/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:55.278063 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51023a-b844-483e-a108-bec0227fac2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=352ca3049c444444a8ce0c29170dd517, user_id=5c9dbce0a6f9482397a79cd734ebab07, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:55.280762 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51023a-b844-483e-a108-bec0227fac2e admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:55.294512 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1448/2896] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:40:55 2022] PUT /identity/v3/projects/352ca3049c444444a8ce0c29170dd517/users/5c9dbce0a6f9482397a79cd734ebab07/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:40:55.304873 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5f701-95d2-4060-a41f-19dba050ee02 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:55.305159 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5f701-95d2-4060-a41f-19dba050ee02 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:55.305430 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5f701-95d2-4060-a41f-19dba050ee02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:55.332790 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-06c5f701-95d2-4060-a41f-19dba050ee02 None None] MFA Rules not processed for user `5c9dbce0a6f9482397a79cd734ebab07`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:55.334003 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-06c5f701-95d2-4060-a41f-19dba050ee02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:55.397415 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1449/2897] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:55.408378 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51023a-b844-483e-a108-bec0227fac2e admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:55.410192 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-2c51023a-b844-483e-a108-bec0227fac2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:55.487075 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51023a-b844-483e-a108-bec0227fac2e admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:55.490775 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7870c18-4f30-48f6-8fe0-8171234e35cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43e873f881e6497ea09f7f10f08e3bd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:55.491373 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7870c18-4f30-48f6-8fe0-8171234e35cb service cinder] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:55.491684 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7870c18-4f30-48f6-8fe0-8171234e35cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:55.491954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7870c18-4f30-48f6-8fe0-8171234e35cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:55.500131 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7870c18-4f30-48f6-8fe0-8171234e35cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:55.501532 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7870c18-4f30-48f6-8fe0-8171234e35cb service cinder] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:55.504953 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:40:55.511851 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1449/2898] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:40:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:40:56.861870 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3f802-aa24-464c-b966-6dba8407cf41 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:56.862379 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3f802-aa24-464c-b966-6dba8407cf41 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:56.862496 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3f802-aa24-464c-b966-6dba8407cf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:56.881222 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-40f3f802-aa24-464c-b966-6dba8407cf41 None None] MFA Rules not processed for user `5c9dbce0a6f9482397a79cd734ebab07`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:40:56.882447 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-40f3f802-aa24-464c-b966-6dba8407cf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:56.889281 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1450/2899] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:40:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:40:58.830974 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7870c18-4f30-48f6-8fe0-8171234e35cb service cinder] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:40:58.832487 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-e7870c18-4f30-48f6-8fe0-8171234e35cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:58.900385 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7870c18-4f30-48f6-8fe0-8171234e35cb service cinder] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:40:58.905993 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd1773-f6a3-4fae-b0bb-e7cc91a17a8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:40:58.906594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd1773-f6a3-4fae-b0bb-e7cc91a17a8f service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:40:58.906877 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd1773-f6a3-4fae-b0bb-e7cc91a17a8f service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:40:58.907150 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd1773-f6a3-4fae-b0bb-e7cc91a17a8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:40:58.908890 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-59fd1773-f6a3-4fae-b0bb-e7cc91a17a8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:40:58.920848 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd1773-f6a3-4fae-b0bb-e7cc91a17a8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:40:58.922416 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd1773-f6a3-4fae-b0bb-e7cc91a17a8f service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:40:58.932152 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1450/2900] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:40:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:41:01.377825 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3f802-aa24-464c-b966-6dba8407cf41 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:01.379309 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-40f3f802-aa24-464c-b966-6dba8407cf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:01.478269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:41:01.489095 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3f802-aa24-464c-b966-6dba8407cf41 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:01.494757 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a65036-b20e-4397-ae5a-8064e2539279 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:01.495413 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a65036-b20e-4397-ae5a-8064e2539279 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:01.495746 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a65036-b20e-4397-ae5a-8064e2539279 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:01.496016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a65036-b20e-4397-ae5a-8064e2539279 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:01.497713 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-62a65036-b20e-4397-ae5a-8064e2539279 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:01.510080 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a65036-b20e-4397-ae5a-8064e2539279 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:01.511829 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a65036-b20e-4397-ae5a-8064e2539279 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:01.522288 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1451/2901] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:41:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:41:08.927896 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=33 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:41:11.517848 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=7 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:41:13.919319 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80439de5-716d-4254-a33b-a6b6e6fd586a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:13.919826 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80439de5-716d-4254-a33b-a6b6e6fd586a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:13.919935 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80439de5-716d-4254-a33b-a6b6e6fd586a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:13.949507 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-80439de5-716d-4254-a33b-a6b6e6fd586a None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:41:13.950688 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-80439de5-716d-4254-a33b-a6b6e6fd586a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:14.004255 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1451/2902] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Oct 6 12:41:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:41:14.050216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a65036-b20e-4397-ae5a-8064e2539279 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:14.051646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-62a65036-b20e-4397-ae5a-8064e2539279 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:14.117038 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a65036-b20e-4397-ae5a-8064e2539279 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:14.121961 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b055b69-e55f-459a-ab16-9345ec7d2196 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:14.122551 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b055b69-e55f-459a-ab16-9345ec7d2196 service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:14.122826 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b055b69-e55f-459a-ab16-9345ec7d2196 service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:14.123093 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b055b69-e55f-459a-ab16-9345ec7d2196 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:14.132593 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b055b69-e55f-459a-ab16-9345ec7d2196 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:14.134196 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b055b69-e55f-459a-ab16-9345ec7d2196 service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:14.142251 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1452/2903] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:41:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 12:41:14.629615 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd5913-2c66-4936-9a95-87535bdf0ebd None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:14.629911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd5913-2c66-4936-9a95-87535bdf0ebd None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:14.630218 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd5913-2c66-4936-9a95-87535bdf0ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:14.649681 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-0cdd5913-2c66-4936-9a95-87535bdf0ebd None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:41:14.650913 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-0cdd5913-2c66-4936-9a95-87535bdf0ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:14.659103 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1452/2904] 10.222.0.33 () {58 vars in 967 bytes} [Thu Oct 6 12:41:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:41:14.671350 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b055b69-e55f-459a-ab16-9345ec7d2196 service swift] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:14.681536 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b055b69-e55f-459a-ab16-9345ec7d2196 service swift] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:14.686408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a8c9a-aecc-406d-b10b-1ffbaec04c1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:14.686982 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a8c9a-aecc-406d-b10b-1ffbaec04c1c service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:14.687259 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a8c9a-aecc-406d-b10b-1ffbaec04c1c service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:14.687560 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a8c9a-aecc-406d-b10b-1ffbaec04c1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:14.695214 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a8c9a-aecc-406d-b10b-1ffbaec04c1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:14.696798 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a8c9a-aecc-406d-b10b-1ffbaec04c1c service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:14.705082 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1453/2905] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:41:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 12:41:14.926060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc87adb-0c2f-4b79-9f7b-a56ee76ffffc None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:14.926553 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc87adb-0c2f-4b79-9f7b-a56ee76ffffc None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:14.926664 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc87adb-0c2f-4b79-9f7b-a56ee76ffffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:14.945954 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-7fc87adb-0c2f-4b79-9f7b-a56ee76ffffc None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:41:14.947145 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7fc87adb-0c2f-4b79-9f7b-a56ee76ffffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:14.954776 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1453/2906] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Oct 6 12:41:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:41:14.966601 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a8c9a-aecc-406d-b10b-1ffbaec04c1c service swift] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:14.977208 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a8c9a-aecc-406d-b10b-1ffbaec04c1c service swift] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:14.982269 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd28ec-fc3f-4d34-bdd1-13f1c909fd0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:14.982864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd28ec-fc3f-4d34-bdd1-13f1c909fd0b service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:14.983173 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd28ec-fc3f-4d34-bdd1-13f1c909fd0b service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:14.983447 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd28ec-fc3f-4d34-bdd1-13f1c909fd0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:14.991637 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd28ec-fc3f-4d34-bdd1-13f1c909fd0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:14.993204 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd28ec-fc3f-4d34-bdd1-13f1c909fd0b service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:15.001521 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1454/2907] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:41:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 12:41:15.314916 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e412a0-9681-4eac-8a59-c65473192840 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:15.315198 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e412a0-9681-4eac-8a59-c65473192840 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:15.315466 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e412a0-9681-4eac-8a59-c65473192840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:15.334453 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-21e412a0-9681-4eac-8a59-c65473192840 None None] MFA Rules not processed for user `96a302378dc0416d87c1cf11a3ba0a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:41:15.335767 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-21e412a0-9681-4eac-8a59-c65473192840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:15.342763 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1454/2908] 10.222.0.33 () {58 vars in 967 bytes} [Thu Oct 6 12:41:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:41:15.354630 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd28ec-fc3f-4d34-bdd1-13f1c909fd0b service swift] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:15.365626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd28ec-fc3f-4d34-bdd1-13f1c909fd0b service swift] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:15.370965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b2e9a-7d7f-4d6d-8e9f-782cddd50f6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3deb4b248f884c19bf1e444f2fe0923a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:15.371368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b2e9a-7d7f-4d6d-8e9f-782cddd50f6d service swift] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:15.372231 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b2e9a-7d7f-4d6d-8e9f-782cddd50f6d service swift] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:15.372571 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b2e9a-7d7f-4d6d-8e9f-782cddd50f6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:15.380626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b2e9a-7d7f-4d6d-8e9f-782cddd50f6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:15.382179 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b2e9a-7d7f-4d6d-8e9f-782cddd50f6d service swift] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:15.389976 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1455/2909] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Oct 6 12:41:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 12:41:17.619254 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e412a0-9681-4eac-8a59-c65473192840 None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:17.680258 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e412a0-9681-4eac-8a59-c65473192840 None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:17.684655 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874186f-95b7-4cff-ac03-a2d13bd43d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:17.685827 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874186f-95b7-4cff-ac03-a2d13bd43d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:17.686107 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874186f-95b7-4cff-ac03-a2d13bd43d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:17.686377 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874186f-95b7-4cff-ac03-a2d13bd43d0f admin admin] PATH_INFO: `/v3/users/5c9dbce0a6f9482397a79cd734ebab07` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:17.687488 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1874186f-95b7-4cff-ac03-a2d13bd43d0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c9dbce0a6f9482397a79cd734ebab07)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:17.689404 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1874186f-95b7-4cff-ac03-a2d13bd43d0f admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:17.734222 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1874186f-95b7-4cff-ac03-a2d13bd43d0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c9dbce0a6f9482397a79cd734ebab07'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:41:17.741502 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1874186f-95b7-4cff-ac03-a2d13bd43d0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c9dbce0a6f9482397a79cd734ebab07'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:41:17.742165 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1874186f-95b7-4cff-ac03-a2d13bd43d0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c9dbce0a6f9482397a79cd734ebab07'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:41:17.742785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1874186f-95b7-4cff-ac03-a2d13bd43d0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c9dbce0a6f9482397a79cd734ebab07'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:41:17.749661 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1874186f-95b7-4cff-ac03-a2d13bd43d0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c9dbce0a6f9482397a79cd734ebab07'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:41:17.760484 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:41:17.768073 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1455/2910] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:41:17 2022] DELETE /identity/v3/users/5c9dbce0a6f9482397a79cd734ebab07 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:41:17.772555 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ede9b-b799-42aa-895b-157a58341ae4 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:17.773056 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ede9b-b799-42aa-895b-157a58341ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:17.773174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ede9b-b799-42aa-895b-157a58341ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:17.792348 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-114ede9b-b799-42aa-895b-157a58341ae4 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:41:17.793542 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-114ede9b-b799-42aa-895b-157a58341ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:17.851094 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1456/2911] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:41:17.861324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874186f-95b7-4cff-ac03-a2d13bd43d0f admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:17.862733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1874186f-95b7-4cff-ac03-a2d13bd43d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:17.918312 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874186f-95b7-4cff-ac03-a2d13bd43d0f admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:17.922714 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395fcba-ff3f-439b-9ec1-93f0e52f434a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:17.923309 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395fcba-ff3f-439b-9ec1-93f0e52f434a service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:17.923622 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395fcba-ff3f-439b-9ec1-93f0e52f434a service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:17.923899 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395fcba-ff3f-439b-9ec1-93f0e52f434a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:17.938905 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395fcba-ff3f-439b-9ec1-93f0e52f434a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:17.941030 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395fcba-ff3f-439b-9ec1-93f0e52f434a service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:17.950430 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1456/2912] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:41:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:41:18.115368 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ede9b-b799-42aa-895b-157a58341ae4 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:18.116906 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-114ede9b-b799-42aa-895b-157a58341ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:18.128988 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ede9b-b799-42aa-895b-157a58341ae4 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:18.134013 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9827fb-bc77-4219-aa69-fbefbc9e4456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:18.134660 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9827fb-bc77-4219-aa69-fbefbc9e4456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:18.134942 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9827fb-bc77-4219-aa69-fbefbc9e4456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:18.135218 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9827fb-bc77-4219-aa69-fbefbc9e4456 admin admin] PATH_INFO: `/v3/projects/352ca3049c444444a8ce0c29170dd517` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:18.136397 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9827fb-bc77-4219-aa69-fbefbc9e4456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=352ca3049c444444a8ce0c29170dd517)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:18.138822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9827fb-bc77-4219-aa69-fbefbc9e4456 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:18.183133 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:41:18.186405 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c9827fb-bc77-4219-aa69-fbefbc9e4456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '352ca3049c444444a8ce0c29170dd517', 'request_id': 'req-8c9827fb-bc77-4219-aa69-fbefbc9e4456'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:41:18.194062 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c9827fb-bc77-4219-aa69-fbefbc9e4456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '352ca3049c444444a8ce0c29170dd517', 'request_id': 'req-8c9827fb-bc77-4219-aa69-fbefbc9e4456'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:41:18.200250 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c9827fb-bc77-4219-aa69-fbefbc9e4456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '352ca3049c444444a8ce0c29170dd517', 'request_id': 'req-8c9827fb-bc77-4219-aa69-fbefbc9e4456'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:41:18.210854 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c9827fb-bc77-4219-aa69-fbefbc9e4456 admin admin] The token cache is being invalidate because project 352ca3049c444444a8ce0c29170dd517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:41:18.211140 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c9827fb-bc77-4219-aa69-fbefbc9e4456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:41:18.211780 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-8c9827fb-bc77-4219-aa69-fbefbc9e4456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:41:18.213233 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1457/2913] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:41:18 2022] DELETE /identity/v3/projects/352ca3049c444444a8ce0c29170dd517 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:41:18.218701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87cdde-6f14-4063-b575-3d6e719cc90a None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:18.218988 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87cdde-6f14-4063-b575-3d6e719cc90a None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:18.219273 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87cdde-6f14-4063-b575-3d6e719cc90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:18.238113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-de87cdde-6f14-4063-b575-3d6e719cc90a None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:41:18.239302 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-de87cdde-6f14-4063-b575-3d6e719cc90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:18.301024 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1457/2914] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:41:18.308677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6837d73-2ec5-4e4a-b902-5d4e54f31a17 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:18.308965 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6837d73-2ec5-4e4a-b902-5d4e54f31a17 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:18.309240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6837d73-2ec5-4e4a-b902-5d4e54f31a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:18.329373 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-a6837d73-2ec5-4e4a-b902-5d4e54f31a17 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:41:18.330574 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-a6837d73-2ec5-4e4a-b902-5d4e54f31a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:18.338419 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1458/2915] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:41:18.343353 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87cdde-6f14-4063-b575-3d6e719cc90a None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:18.352906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87cdde-6f14-4063-b575-3d6e719cc90a None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:18.357815 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475bcc4-1a7d-43b7-be64-519a5d2984c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:18.358347 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475bcc4-1a7d-43b7-be64-519a5d2984c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:18.358631 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475bcc4-1a7d-43b7-be64-519a5d2984c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:18.358907 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475bcc4-1a7d-43b7-be64-519a5d2984c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:18.359613 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475bcc4-1a7d-43b7-be64-519a5d2984c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 12:41:18.359923 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475bcc4-1a7d-43b7-be64-519a5d2984c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:18.361536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475bcc4-1a7d-43b7-be64-519a5d2984c2 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:18.373074 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1458/2916] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Oct 6 12:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 12:41:18.377170 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6837d73-2ec5-4e4a-b902-5d4e54f31a17 None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:18.386185 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6837d73-2ec5-4e4a-b902-5d4e54f31a17 None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:18.391282 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3243250-96dd-49b7-9375-03a57855d2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:18.391900 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3243250-96dd-49b7-9375-03a57855d2c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:18.392182 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3243250-96dd-49b7-9375-03a57855d2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:18.392458 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3243250-96dd-49b7-9375-03a57855d2c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:18.393083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3243250-96dd-49b7-9375-03a57855d2c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:18.394964 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3243250-96dd-49b7-9375-03a57855d2c5 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:18.422006 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1459/2917] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:41:18 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:41:18.426113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475bcc4-1a7d-43b7-be64-519a5d2984c2 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:18.436600 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475bcc4-1a7d-43b7-be64-519a5d2984c2 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:18.491349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce9010-f567-4bbd-a5d9-ee526584fa33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:18.491982 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce9010-f567-4bbd-a5d9-ee526584fa33 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:18.492264 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce9010-f567-4bbd-a5d9-ee526584fa33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:18.492536 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce9010-f567-4bbd-a5d9-ee526584fa33 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:18.493162 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce9010-f567-4bbd-a5d9-ee526584fa33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:18.495098 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce9010-f567-4bbd-a5d9-ee526584fa33 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:18.516674 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1459/2918] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:41:18 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:41:18.520800 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3243250-96dd-49b7-9375-03a57855d2c5 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:18.530832 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3243250-96dd-49b7-9375-03a57855d2c5 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:18.535335 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db777333-88d3-4525-b163-bbd1ccdef1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:18.536238 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db777333-88d3-4525-b163-bbd1ccdef1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:18.536511 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db777333-88d3-4525-b163-bbd1ccdef1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:18.536776 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db777333-88d3-4525-b163-bbd1ccdef1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:18.537408 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db777333-88d3-4525-b163-bbd1ccdef1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:18.538996 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db777333-88d3-4525-b163-bbd1ccdef1a2 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:18.549060 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1460/2919] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 12:41:18.552787 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce9010-f567-4bbd-a5d9-ee526584fa33 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:18.563275 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce9010-f567-4bbd-a5d9-ee526584fa33 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:18.568041 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94247fa-d740-41ec-8865-2d3ccb6c3aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:18.568696 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94247fa-d740-41ec-8865-2d3ccb6c3aab admin admin] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:18.568907 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94247fa-d740-41ec-8865-2d3ccb6c3aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:18.569188 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94247fa-d740-41ec-8865-2d3ccb6c3aab admin admin] PATH_INFO: `/v3/roles` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:18.569833 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94247fa-d740-41ec-8865-2d3ccb6c3aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:18.571522 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94247fa-d740-41ec-8865-2d3ccb6c3aab admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:18.582844 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1460/2920] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:41:18.586864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db777333-88d3-4525-b163-bbd1ccdef1a2 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:18.596735 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db777333-88d3-4525-b163-bbd1ccdef1a2 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:18.601385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ce19b-4aa2-405d-8dfc-beefd1a87441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:18.602024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ce19b-4aa2-405d-8dfc-beefd1a87441 admin admin] REQUEST_METHOD: `PUT` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:18.602303 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ce19b-4aa2-405d-8dfc-beefd1a87441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:18.602575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ce19b-4aa2-405d-8dfc-beefd1a87441 admin admin] PATH_INFO: `/v3/projects/b094fe65c1d94c368068f0ebd9eee85a/users/e8cf9f6b209240339053a939bcdf1793/roles/bbea734b8a704d7ca8ec61bd45536cbe` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:18.616772 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ce19b-4aa2-405d-8dfc-beefd1a87441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b094fe65c1d94c368068f0ebd9eee85a, user_id=e8cf9f6b209240339053a939bcdf1793, role_id=bbea734b8a704d7ca8ec61bd45536cbe)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:18.619383 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ce19b-4aa2-405d-8dfc-beefd1a87441 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:18.633323 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1461/2921] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:41:18 2022] PUT /identity/v3/projects/b094fe65c1d94c368068f0ebd9eee85a/users/e8cf9f6b209240339053a939bcdf1793/roles/bbea734b8a704d7ca8ec61bd45536cbe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:41:18.638910 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f4ab4-6fe9-45f0-9796-e9c3c6901a83 None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:18.639390 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f4ab4-6fe9-45f0-9796-e9c3c6901a83 None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:18.639534 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f4ab4-6fe9-45f0-9796-e9c3c6901a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:18.647920 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:41:18.659822 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-7e3f4ab4-6fe9-45f0-9796-e9c3c6901a83 None None] MFA Rules not processed for user `50b3556af8274b8a9a59fe39e12271c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:41:18.661301 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-7e3f4ab4-6fe9-45f0-9796-e9c3c6901a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:18.718892 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1461/2922] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:41:18.728362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ce19b-4aa2-405d-8dfc-beefd1a87441 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:18.730065 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-8f0ce19b-4aa2-405d-8dfc-beefd1a87441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:18.788063 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ce19b-4aa2-405d-8dfc-beefd1a87441 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:18.791750 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017de8f-4e2b-4fee-a447-b54263e18814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:18.792351 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017de8f-4e2b-4fee-a447-b54263e18814 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:18.792628 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017de8f-4e2b-4fee-a447-b54263e18814 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:18.792896 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017de8f-4e2b-4fee-a447-b54263e18814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:18.802489 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017de8f-4e2b-4fee-a447-b54263e18814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:18.803937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017de8f-4e2b-4fee-a447-b54263e18814 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:18.813698 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1462/2923] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:41:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:41:22.772322 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78387bac-fb95-4cbd-8be9-4d28b361980f None None] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:22.772810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78387bac-fb95-4cbd-8be9-4d28b361980f None None] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:22.772902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78387bac-fb95-4cbd-8be9-4d28b361980f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:22.802916 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.auth.core [None req-78387bac-fb95-4cbd-8be9-4d28b361980f None None] MFA Rules not processed for user `e8cf9f6b209240339053a939bcdf1793`. Rule list: `[]` (Enabled: `True`). {{(pid=81355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:41:22.804170 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-78387bac-fb95-4cbd-8be9-4d28b361980f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:22.873898 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1462/2924] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:41:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:41:22.878502 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017de8f-4e2b-4fee-a447-b54263e18814 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:22.889258 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017de8f-4e2b-4fee-a447-b54263e18814 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:22.893997 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baddf9e-8c4a-4b39-a1c1-fd2a8ce92c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:22.894604 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baddf9e-8c4a-4b39-a1c1-fd2a8ce92c0b admin admin] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:22.894893 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baddf9e-8c4a-4b39-a1c1-fd2a8ce92c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:22.895172 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baddf9e-8c4a-4b39-a1c1-fd2a8ce92c0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:22.895849 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baddf9e-8c4a-4b39-a1c1-fd2a8ce92c0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:22.897765 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baddf9e-8c4a-4b39-a1c1-fd2a8ce92c0b admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:22.927174 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1463/2925] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Oct 6 12:41:22 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 12:41:22.929481 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78387bac-fb95-4cbd-8be9-4d28b361980f None None] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:22.940834 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78387bac-fb95-4cbd-8be9-4d28b361980f None None] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:22.996304 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270fc26-2e62-4ede-bbde-d12c28e9fab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:22.996831 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270fc26-2e62-4ede-bbde-d12c28e9fab8 admin admin] REQUEST_METHOD: `POST` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:22.997656 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270fc26-2e62-4ede-bbde-d12c28e9fab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:22.997930 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270fc26-2e62-4ede-bbde-d12c28e9fab8 admin admin] PATH_INFO: `/v3/users` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:22.998566 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270fc26-2e62-4ede-bbde-d12c28e9fab8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:23.000629 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270fc26-2e62-4ede-bbde-d12c28e9fab8 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:23.021804 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1463/2926] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Oct 6 12:41:22 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 12:41:23.025837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baddf9e-8c4a-4b39-a1c1-fd2a8ce92c0b admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:23.036184 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baddf9e-8c4a-4b39-a1c1-fd2a8ce92c0b admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:23.040793 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ce341-6d8e-4af1-bc5c-30dc7e072c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:23.041385 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ce341-6d8e-4af1-bc5c-30dc7e072c56 admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:23.041665 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ce341-6d8e-4af1-bc5c-30dc7e072c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:23.041937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ce341-6d8e-4af1-bc5c-30dc7e072c56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:23.042568 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ce341-6d8e-4af1-bc5c-30dc7e072c56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:23.044194 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ce341-6d8e-4af1-bc5c-30dc7e072c56 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:23.054612 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1464/2927] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:41:23.058662 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270fc26-2e62-4ede-bbde-d12c28e9fab8 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:23.069478 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270fc26-2e62-4ede-bbde-d12c28e9fab8 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:23.074362 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebefab-36b3-4bdc-96f8-c7ea45a0190b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:23.075028 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebefab-36b3-4bdc-96f8-c7ea45a0190b admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:23.075906 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebefab-36b3-4bdc-96f8-c7ea45a0190b admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:23.076192 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebefab-36b3-4bdc-96f8-c7ea45a0190b admin admin] PATH_INFO: `/v3/projects/607dba9743514411aa4c3a80595a42f3/users/af7088723a894b90a1e47f0dd8c9c603/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:23.091984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebefab-36b3-4bdc-96f8-c7ea45a0190b admin admin] RBAC: Authorizing `identity:create_grant(project_id=607dba9743514411aa4c3a80595a42f3, user_id=af7088723a894b90a1e47f0dd8c9c603, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:23.094611 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebefab-36b3-4bdc-96f8-c7ea45a0190b admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:23.103080 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:41:23.108216 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1464/2928] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:41:23 2022] PUT /identity/v3/projects/607dba9743514411aa4c3a80595a42f3/users/af7088723a894b90a1e47f0dd8c9c603/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:41:23.112143 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ce341-6d8e-4af1-bc5c-30dc7e072c56 admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:23.123061 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ce341-6d8e-4af1-bc5c-30dc7e072c56 admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:23.164360 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:41:23.183344 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf79468-9e08-4332-bc68-5fbedf9fc2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:23.183972 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf79468-9e08-4332-bc68-5fbedf9fc2eb admin admin] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:23.184255 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf79468-9e08-4332-bc68-5fbedf9fc2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:23.185028 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf79468-9e08-4332-bc68-5fbedf9fc2eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:23.185657 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf79468-9e08-4332-bc68-5fbedf9fc2eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:23.187371 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf79468-9e08-4332-bc68-5fbedf9fc2eb admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:23.196729 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1465/2929] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Oct 6 12:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 12:41:23.201470 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebefab-36b3-4bdc-96f8-c7ea45a0190b admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:23.211728 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebefab-36b3-4bdc-96f8-c7ea45a0190b admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:23.216327 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091681c8-bbfe-4792-8c6b-85e06b333baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:23.216968 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091681c8-bbfe-4792-8c6b-85e06b333baf admin admin] REQUEST_METHOD: `PUT` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:23.217251 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091681c8-bbfe-4792-8c6b-85e06b333baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:23.217525 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091681c8-bbfe-4792-8c6b-85e06b333baf admin admin] PATH_INFO: `/v3/projects/607dba9743514411aa4c3a80595a42f3/users/af7088723a894b90a1e47f0dd8c9c603/roles/d5ddd52de5714fabbef9404d38ef4b35` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:23.219436 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091681c8-bbfe-4792-8c6b-85e06b333baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=607dba9743514411aa4c3a80595a42f3, user_id=af7088723a894b90a1e47f0dd8c9c603, role_id=d5ddd52de5714fabbef9404d38ef4b35)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:23.222070 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091681c8-bbfe-4792-8c6b-85e06b333baf admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:23.232501 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1465/2930] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Oct 6 12:41:23 2022] PUT /identity/v3/projects/607dba9743514411aa4c3a80595a42f3/users/af7088723a894b90a1e47f0dd8c9c603/roles/d5ddd52de5714fabbef9404d38ef4b35 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:41:27.658725 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb092e0-c986-426d-b4aa-93089b839c24 None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:27.659216 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb092e0-c986-426d-b4aa-93089b839c24 None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:27.659338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb092e0-c986-426d-b4aa-93089b839c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:27.690864 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-7fb092e0-c986-426d-b4aa-93089b839c24 None None] MFA Rules not processed for user `af7088723a894b90a1e47f0dd8c9c603`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:41:27.692432 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-7fb092e0-c986-426d-b4aa-93089b839c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:27.775801 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1466/2931] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:41:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:41:27.791223 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091681c8-bbfe-4792-8c6b-85e06b333baf admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:27.794567 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-091681c8-bbfe-4792-8c6b-85e06b333baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:27.893737 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091681c8-bbfe-4792-8c6b-85e06b333baf admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:27.899807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da01ac-1ee7-4d83-a6ef-0b77aff09001 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:27.900136 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da01ac-1ee7-4d83-a6ef-0b77aff09001 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:27.900422 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da01ac-1ee7-4d83-a6ef-0b77aff09001 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:27.900697 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da01ac-1ee7-4d83-a6ef-0b77aff09001 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:27.964927 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da01ac-1ee7-4d83-a6ef-0b77aff09001 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:27.967142 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da01ac-1ee7-4d83-a6ef-0b77aff09001 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:27.977101 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1466/2932] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:41:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:41:28.954332 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c441205-e7d4-4552-8036-d6d8cd488a3f None None] REQUEST_METHOD: `POST` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:28.954836 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c441205-e7d4-4552-8036-d6d8cd488a3f None None] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:28.954957 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c441205-e7d4-4552-8036-d6d8cd488a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:28.973956 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.auth.core [None req-6c441205-e7d4-4552-8036-d6d8cd488a3f None None] MFA Rules not processed for user `e8cf9f6b209240339053a939bcdf1793`. Rule list: `[]` (Enabled: `True`). {{(pid=81354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 12:41:28.975152 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-6c441205-e7d4-4552-8036-d6d8cd488a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:28.981999 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1467/2933] 10.222.0.33 () {58 vars in 958 bytes} [Thu Oct 6 12:41:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 12:41:37.973417 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=61,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:41:38.969056 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=48 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:41:50.810008 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da01ac-1ee7-4d83-a6ef-0b77aff09001 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:41:50.811545 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-66da01ac-1ee7-4d83-a6ef-0b77aff09001 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:50.868021 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da01ac-1ee7-4d83-a6ef-0b77aff09001 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:41:50.871619 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54a044-1ed1-48fe-9433-361a55cdfd34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:41:50.872220 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54a044-1ed1-48fe-9433-361a55cdfd34 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:41:50.872498 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54a044-1ed1-48fe-9433-361a55cdfd34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:41:50.872770 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54a044-1ed1-48fe-9433-361a55cdfd34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:41:50.874515 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-4e54a044-1ed1-48fe-9433-361a55cdfd34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:41:50.884025 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54a044-1ed1-48fe-9433-361a55cdfd34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:41:50.885428 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54a044-1ed1-48fe-9433-361a55cdfd34 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:41:50.893594 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1467/2934] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:41:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:42:00.889911 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=26 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:42:08.208181 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c441205-e7d4-4552-8036-d6d8cd488a3f None None] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:42:08.217883 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c441205-e7d4-4552-8036-d6d8cd488a3f None None] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:42:08.221423 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79537794-ec20-4543-a924-cb5b85a11e72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:42:08.222018 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79537794-ec20-4543-a924-cb5b85a11e72 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:42:08.222299 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79537794-ec20-4543-a924-cb5b85a11e72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:42:08.222574 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79537794-ec20-4543-a924-cb5b85a11e72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:42:08.231339 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79537794-ec20-4543-a924-cb5b85a11e72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:42:08.232806 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79537794-ec20-4543-a924-cb5b85a11e72 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:42:08.242646 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1468/2935] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:42:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:42:18.238454 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:43:17.952089 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54a044-1ed1-48fe-9433-361a55cdfd34 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:43:17.961212 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54a044-1ed1-48fe-9433-361a55cdfd34 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:43:17.964765 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65034496-daec-4cb7-984d-2efcecd3f5e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:43:17.965361 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65034496-daec-4cb7-984d-2efcecd3f5e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:43:17.965659 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65034496-daec-4cb7-984d-2efcecd3f5e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:43:17.965926 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65034496-daec-4cb7-984d-2efcecd3f5e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:43:17.967648 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-65034496-daec-4cb7-984d-2efcecd3f5e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:43:17.977569 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65034496-daec-4cb7-984d-2efcecd3f5e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:43:17.978965 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65034496-daec-4cb7-984d-2efcecd3f5e0 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:43:17.986902 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1468/2936] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:43:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:43:27.983313 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=6 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:44:25.419221 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79537794-ec20-4543-a924-cb5b85a11e72 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:44:25.428926 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79537794-ec20-4543-a924-cb5b85a11e72 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:44:25.432649 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13218a90-65d2-4753-aecd-e1d097773bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:44:25.433240 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13218a90-65d2-4753-aecd-e1d097773bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:44:25.433524 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13218a90-65d2-4753-aecd-e1d097773bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:44:25.433799 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13218a90-65d2-4753-aecd-e1d097773bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:44:25.435493 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-13218a90-65d2-4753-aecd-e1d097773bf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:44:25.449585 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13218a90-65d2-4753-aecd-e1d097773bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:44:25.451022 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13218a90-65d2-4753-aecd-e1d097773bf1 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:44:25.461840 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1469/2937] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:44:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 12:44:26.638730 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65034496-daec-4cb7-984d-2efcecd3f5e0 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:44:26.647300 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65034496-daec-4cb7-984d-2efcecd3f5e0 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:44:26.650855 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44cde1-52d9-4fc6-a701-5ec8bbf2b4d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:44:26.651440 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44cde1-52d9-4fc6-a701-5ec8bbf2b4d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:44:26.651772 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44cde1-52d9-4fc6-a701-5ec8bbf2b4d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:44:26.652043 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44cde1-52d9-4fc6-a701-5ec8bbf2b4d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:44:26.653701 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-6c44cde1-52d9-4fc6-a701-5ec8bbf2b4d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:44:26.667459 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44cde1-52d9-4fc6-a701-5ec8bbf2b4d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:44:26.668889 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44cde1-52d9-4fc6-a701-5ec8bbf2b4d7 service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:44:26.680217 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1469/2938] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:44:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:44:35.456941 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=6 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:44:36.674707 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=6 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:44:38.649197 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13218a90-65d2-4753-aecd-e1d097773bf1 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:44:38.657620 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13218a90-65d2-4753-aecd-e1d097773bf1 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:44:38.661449 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8673cc-f50b-4db0-8698-5ca92a979395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:44:38.662141 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8673cc-f50b-4db0-8698-5ca92a979395 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:44:38.663000 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8673cc-f50b-4db0-8698-5ca92a979395 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:44:38.663287 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8673cc-f50b-4db0-8698-5ca92a979395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:44:38.665083 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-da8673cc-f50b-4db0-8698-5ca92a979395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:44:38.674608 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8673cc-f50b-4db0-8698-5ca92a979395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:44:38.676058 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8673cc-f50b-4db0-8698-5ca92a979395 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:44:38.684190 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1470/2939] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:44:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:44:38.803675 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44cde1-52d9-4fc6-a701-5ec8bbf2b4d7 service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:44:38.815105 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44cde1-52d9-4fc6-a701-5ec8bbf2b4d7 service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:44:38.819882 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260a3e4-c9e3-4d1f-83ea-0f0f2aba61ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:44:38.820733 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260a3e4-c9e3-4d1f-83ea-0f0f2aba61ee service neutron] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:44:38.821018 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260a3e4-c9e3-4d1f-83ea-0f0f2aba61ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:44:38.821294 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260a3e4-c9e3-4d1f-83ea-0f0f2aba61ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:44:38.823070 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5260a3e4-c9e3-4d1f-83ea-0f0f2aba61ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:44:38.833739 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260a3e4-c9e3-4d1f-83ea-0f0f2aba61ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:44:38.835744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260a3e4-c9e3-4d1f-83ea-0f0f2aba61ee service neutron] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:44:38.844324 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1470/2940] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:44:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:44:38.952837 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8673cc-f50b-4db0-8698-5ca92a979395 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:44:38.962422 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8673cc-f50b-4db0-8698-5ca92a979395 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:44:38.966210 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b063d1-4206-445a-8809-5dd547406117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:44:38.966939 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b063d1-4206-445a-8809-5dd547406117 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:44:38.972159 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b063d1-4206-445a-8809-5dd547406117 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:44:38.972159 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b063d1-4206-445a-8809-5dd547406117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:44:38.972159 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-49b063d1-4206-445a-8809-5dd547406117 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:44:38.983869 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b063d1-4206-445a-8809-5dd547406117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:44:38.985284 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b063d1-4206-445a-8809-5dd547406117 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:44:38.996101 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1471/2941] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:44:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:44:39.059699 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260a3e4-c9e3-4d1f-83ea-0f0f2aba61ee service neutron] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:44:39.061166 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5260a3e4-c9e3-4d1f-83ea-0f0f2aba61ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:44:39.128410 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260a3e4-c9e3-4d1f-83ea-0f0f2aba61ee service neutron] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:44:39.133845 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233aed2b-5bad-43e9-ad2d-32b4e5e3ba0b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:44:39.134440 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233aed2b-5bad-43e9-ad2d-32b4e5e3ba0b service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:44:39.134717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233aed2b-5bad-43e9-ad2d-32b4e5e3ba0b service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:44:39.134986 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233aed2b-5bad-43e9-ad2d-32b4e5e3ba0b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:44:39.136974 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-233aed2b-5bad-43e9-ad2d-32b4e5e3ba0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:44:39.149156 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233aed2b-5bad-43e9-ad2d-32b4e5e3ba0b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:44:39.150720 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233aed2b-5bad-43e9-ad2d-32b4e5e3ba0b service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:44:39.160791 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1471/2942] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:44:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:44:41.997051 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b063d1-4206-445a-8809-5dd547406117 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:44:42.009498 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b063d1-4206-445a-8809-5dd547406117 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:44:42.014937 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12db44-8ed2-42c2-9346-cfd946c6c60a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:44:42.015570 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12db44-8ed2-42c2-9346-cfd946c6c60a service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:44:42.015851 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12db44-8ed2-42c2-9346-cfd946c6c60a service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:44:42.016117 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12db44-8ed2-42c2-9346-cfd946c6c60a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:44:42.017822 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ec12db44-8ed2-42c2-9346-cfd946c6c60a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:44:42.029923 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12db44-8ed2-42c2-9346-cfd946c6c60a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:44:42.031480 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12db44-8ed2-42c2-9346-cfd946c6c60a service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:44:42.043595 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1472/2943] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:44:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:44:49.156242 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=38 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:44:52.037946 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=18 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:46:24.245393 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233aed2b-5bad-43e9-ad2d-32b4e5e3ba0b service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:46:24.246535 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-233aed2b-5bad-43e9-ad2d-32b4e5e3ba0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:46:24.260113 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233aed2b-5bad-43e9-ad2d-32b4e5e3ba0b service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:46:24.265209 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde62a28-d4bd-45c6-a561-c7c93ca57787 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:46:24.265811 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde62a28-d4bd-45c6-a561-c7c93ca57787 service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:46:24.266090 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde62a28-d4bd-45c6-a561-c7c93ca57787 service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:46:24.266361 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde62a28-d4bd-45c6-a561-c7c93ca57787 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:46:24.268071 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-dde62a28-d4bd-45c6-a561-c7c93ca57787 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:46:24.279182 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde62a28-d4bd-45c6-a561-c7c93ca57787 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:46:24.280790 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde62a28-d4bd-45c6-a561-c7c93ca57787 service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:46:24.290815 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1472/2944] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:46:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:46:26.596171 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12db44-8ed2-42c2-9346-cfd946c6c60a service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:46:26.597603 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ec12db44-8ed2-42c2-9346-cfd946c6c60a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:46:26.612270 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12db44-8ed2-42c2-9346-cfd946c6c60a service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:46:26.618721 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef469f6c-929d-408d-81fa-904edb01b5f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:46:26.619338 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef469f6c-929d-408d-81fa-904edb01b5f2 service nova] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:46:26.619652 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef469f6c-929d-408d-81fa-904edb01b5f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:46:26.619927 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef469f6c-929d-408d-81fa-904edb01b5f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:46:26.621806 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-ef469f6c-929d-408d-81fa-904edb01b5f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:46:26.635310 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef469f6c-929d-408d-81fa-904edb01b5f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:46:26.637238 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef469f6c-929d-408d-81fa-904edb01b5f2 service nova] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:46:26.650032 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1473/2945] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:46:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:46:27.922060 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde62a28-d4bd-45c6-a561-c7c93ca57787 service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:46:27.934315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde62a28-d4bd-45c6-a561-c7c93ca57787 service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:46:27.939843 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d7be8-c41f-4ab4-9a32-1b7a2c66ac82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a46e63d34564d34b6b5752706375798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:46:27.940438 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d7be8-c41f-4ab4-9a32-1b7a2c66ac82 service nova] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:46:27.940717 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d7be8-c41f-4ab4-9a32-1b7a2c66ac82 service nova] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:46:27.940985 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d7be8-c41f-4ab4-9a32-1b7a2c66ac82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:46:27.949802 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d7be8-c41f-4ab4-9a32-1b7a2c66ac82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:46:27.951360 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d7be8-c41f-4ab4-9a32-1b7a2c66ac82 service nova] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:46:27.960194 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1473/2946] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:46:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:46:30.488554 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef469f6c-929d-408d-81fa-904edb01b5f2 service nova] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:46:30.497943 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef469f6c-929d-408d-81fa-904edb01b5f2 service nova] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:46:30.501498 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95464e-7438-4097-8c30-a89793065af0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71f9c1626f4f58916a5dd5c5987ccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:46:30.502081 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95464e-7438-4097-8c30-a89793065af0 service neutron] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:46:30.502357 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95464e-7438-4097-8c30-a89793065af0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:46:30.502626 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95464e-7438-4097-8c30-a89793065af0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:46:30.563444 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95464e-7438-4097-8c30-a89793065af0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:46:30.564818 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95464e-7438-4097-8c30-a89793065af0 service neutron] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:46:30.574679 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1474/2947] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:46:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:46:33.598171 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d7be8-c41f-4ab4-9a32-1b7a2c66ac82 service nova] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:46:33.611616 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d7be8-c41f-4ab4-9a32-1b7a2c66ac82 service nova] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:46:33.617426 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b172042-e033-4d56-a213-847c29dcfad0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:46:33.618211 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b172042-e033-4d56-a213-847c29dcfad0 service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:46:33.618588 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b172042-e033-4d56-a213-847c29dcfad0 service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:46:33.619097 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b172042-e033-4d56-a213-847c29dcfad0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:46:33.621079 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-5b172042-e033-4d56-a213-847c29dcfad0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:46:33.634729 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b172042-e033-4d56-a213-847c29dcfad0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:46:33.636455 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b172042-e033-4d56-a213-847c29dcfad0 service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:46:33.648804 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1474/2948] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:46:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:46:36.679411 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95464e-7438-4097-8c30-a89793065af0 service neutron] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:46:36.691479 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95464e-7438-4097-8c30-a89793065af0 service neutron] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:46:36.696575 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77bc2f-6beb-410f-a919-e53556cc092f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:46:36.697178 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77bc2f-6beb-410f-a919-e53556cc092f service placement] REQUEST_METHOD: `GET` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:46:36.697488 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77bc2f-6beb-410f-a919-e53556cc092f service placement] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:46:36.697761 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77bc2f-6beb-410f-a919-e53556cc092f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:46:36.699475 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-5d77bc2f-6beb-410f-a919-e53556cc092f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:46:36.711391 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77bc2f-6beb-410f-a919-e53556cc092f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:46:36.712970 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77bc2f-6beb-410f-a919-e53556cc092f service placement] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:46:36.723336 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1475/2949] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:46:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:46:36.750984 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b172042-e033-4d56-a213-847c29dcfad0 service placement] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:46:36.768619 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b172042-e033-4d56-a213-847c29dcfad0 service placement] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:46:36.773195 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819110d-8cdb-4711-8579-e3f703242255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:46:36.773810 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819110d-8cdb-4711-8579-e3f703242255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:46:36.774303 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819110d-8cdb-4711-8579-e3f703242255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:46:36.774597 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819110d-8cdb-4711-8579-e3f703242255 admin admin] PATH_INFO: `/v3/users/e8cf9f6b209240339053a939bcdf1793` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:46:36.775740 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819110d-8cdb-4711-8579-e3f703242255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8cf9f6b209240339053a939bcdf1793)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:46:36.777625 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819110d-8cdb-4711-8579-e3f703242255 admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:46:36.822807 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1819110d-8cdb-4711-8579-e3f703242255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8cf9f6b209240339053a939bcdf1793'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:36.829661 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1819110d-8cdb-4711-8579-e3f703242255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8cf9f6b209240339053a939bcdf1793'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:36.830336 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1819110d-8cdb-4711-8579-e3f703242255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8cf9f6b209240339053a939bcdf1793'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:36.830928 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1819110d-8cdb-4711-8579-e3f703242255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8cf9f6b209240339053a939bcdf1793'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:36.838676 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-1819110d-8cdb-4711-8579-e3f703242255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8cf9f6b209240339053a939bcdf1793'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:36.856429 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1475/2950] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:46:36 2022] DELETE /identity/v3/users/e8cf9f6b209240339053a939bcdf1793 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:46:37.025318 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77bc2f-6beb-410f-a919-e53556cc092f service placement] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:46:37.040597 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77bc2f-6beb-410f-a919-e53556cc092f service placement] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:46:37.096016 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296a73d-6d14-4a1b-a8e3-09de967817dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:46:37.096642 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296a73d-6d14-4a1b-a8e3-09de967817dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:46:37.096924 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296a73d-6d14-4a1b-a8e3-09de967817dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:46:37.097199 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296a73d-6d14-4a1b-a8e3-09de967817dd admin admin] PATH_INFO: `/v3/projects/b094fe65c1d94c368068f0ebd9eee85a` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:46:37.098313 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296a73d-6d14-4a1b-a8e3-09de967817dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b094fe65c1d94c368068f0ebd9eee85a)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:46:37.100253 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296a73d-6d14-4a1b-a8e3-09de967817dd admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:46:37.146320 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2296a73d-6d14-4a1b-a8e3-09de967817dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b094fe65c1d94c368068f0ebd9eee85a', 'request_id': 'req-2296a73d-6d14-4a1b-a8e3-09de967817dd'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:37.154987 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2296a73d-6d14-4a1b-a8e3-09de967817dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b094fe65c1d94c368068f0ebd9eee85a', 'request_id': 'req-2296a73d-6d14-4a1b-a8e3-09de967817dd'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:37.162030 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2296a73d-6d14-4a1b-a8e3-09de967817dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b094fe65c1d94c368068f0ebd9eee85a', 'request_id': 'req-2296a73d-6d14-4a1b-a8e3-09de967817dd'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:37.169151 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:46:37.174356 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2296a73d-6d14-4a1b-a8e3-09de967817dd admin admin] The token cache is being invalidate because project b094fe65c1d94c368068f0ebd9eee85a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:46:37.174710 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2296a73d-6d14-4a1b-a8e3-09de967817dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:37.175677 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-2296a73d-6d14-4a1b-a8e3-09de967817dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:37.177701 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1476/2951] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:46:37 2022] DELETE /identity/v3/projects/b094fe65c1d94c368068f0ebd9eee85a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:46:37.181867 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819110d-8cdb-4711-8579-e3f703242255 admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:46:37.183280 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1819110d-8cdb-4711-8579-e3f703242255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:46:37.196129 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819110d-8cdb-4711-8579-e3f703242255 admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:46:37.251593 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166dcc4-bc33-43f4-a829-d65a3d61affa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:46:37.252226 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166dcc4-bc33-43f4-a829-d65a3d61affa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:46:37.252511 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166dcc4-bc33-43f4-a829-d65a3d61affa admin admin] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:46:37.252785 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166dcc4-bc33-43f4-a829-d65a3d61affa admin admin] PATH_INFO: `/v3/users/af7088723a894b90a1e47f0dd8c9c603` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:46:37.253944 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166dcc4-bc33-43f4-a829-d65a3d61affa admin admin] RBAC: Authorizing `identity:delete_user(user_id=af7088723a894b90a1e47f0dd8c9c603)` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:46:37.255890 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166dcc4-bc33-43f4-a829-d65a3d61affa admin admin] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:46:37.293405 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:46:37.303990 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3166dcc4-bc33-43f4-a829-d65a3d61affa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af7088723a894b90a1e47f0dd8c9c603'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:37.310863 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3166dcc4-bc33-43f4-a829-d65a3d61affa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af7088723a894b90a1e47f0dd8c9c603'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:37.311562 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3166dcc4-bc33-43f4-a829-d65a3d61affa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af7088723a894b90a1e47f0dd8c9c603'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:37.312182 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3166dcc4-bc33-43f4-a829-d65a3d61affa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af7088723a894b90a1e47f0dd8c9c603'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:37.319522 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.notifications [None req-3166dcc4-bc33-43f4-a829-d65a3d61affa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af7088723a894b90a1e47f0dd8c9c603'} {{(pid=81355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:37.337744 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1476/2952] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Oct 6 12:46:37 2022] DELETE /identity/v3/users/af7088723a894b90a1e47f0dd8c9c603 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:46:37.510511 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296a73d-6d14-4a1b-a8e3-09de967817dd admin admin] Authenticating user token {{(pid=81354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:46:37.512147 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.fernet_utils [None req-2296a73d-6d14-4a1b-a8e3-09de967817dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:46:37.525472 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296a73d-6d14-4a1b-a8e3-09de967817dd admin admin] Validating token access rules against request {{(pid=81354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:46:37.582994 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb9309-ba0a-4f75-9267-414eb09985b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b3556af8274b8a9a59fe39e12271c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5568e23a42b74b0a8d5f679d197b7521', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:46:37.583666 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb9309-ba0a-4f75-9267-414eb09985b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:46:37.583938 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb9309-ba0a-4f75-9267-414eb09985b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:46:37.584206 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb9309-ba0a-4f75-9267-414eb09985b3 admin admin] PATH_INFO: `/v3/projects/607dba9743514411aa4c3a80595a42f3` {{(pid=81354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:46:37.585305 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb9309-ba0a-4f75-9267-414eb09985b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=607dba9743514411aa4c3a80595a42f3)` {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:46:37.587804 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb9309-ba0a-4f75-9267-414eb09985b3 admin admin] RBAC: Authorization granted {{(pid=81354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:46:37.634429 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5feb9309-ba0a-4f75-9267-414eb09985b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '607dba9743514411aa4c3a80595a42f3', 'request_id': 'req-5feb9309-ba0a-4f75-9267-414eb09985b3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:37.642235 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5feb9309-ba0a-4f75-9267-414eb09985b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '607dba9743514411aa4c3a80595a42f3', 'request_id': 'req-5feb9309-ba0a-4f75-9267-414eb09985b3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:37.648898 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5feb9309-ba0a-4f75-9267-414eb09985b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '607dba9743514411aa4c3a80595a42f3', 'request_id': 'req-5feb9309-ba0a-4f75-9267-414eb09985b3'} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:37.660365 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5feb9309-ba0a-4f75-9267-414eb09985b3 admin admin] The token cache is being invalidate because project 607dba9743514411aa4c3a80595a42f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 12:46:37.660559 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5feb9309-ba0a-4f75-9267-414eb09985b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:37.661215 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG keystone.notifications [None req-5feb9309-ba0a-4f75-9267-414eb09985b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 12:46:37.663024 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: [pid: 81354|app: 0|req: 1477/2953] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Oct 6 12:46:37 2022] DELETE /identity/v3/projects/607dba9743514411aa4c3a80595a42f3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 12:46:47.333610 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=16,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:46:47.657362 dv-cinder-856041-2-iscsi devstack@keystone.service[81354]: DEBUG dbcounter [-] [81354] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 12:51:28.216932 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166dcc4-bc33-43f4-a829-d65a3d61affa admin admin] Authenticating user token {{(pid=81355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 12:51:28.218349 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-3166dcc4-bc33-43f4-a829-d65a3d61affa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:51:28.355620 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166dcc4-bc33-43f4-a829-d65a3d61affa admin admin] Validating token access rules against request {{(pid=81355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 12:51:28.417315 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80a758-7818-46dd-b87c-0ff9b6675ecc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39d67f5edf4b441ebaa2bb42e74b2cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '314d3d13ba2e4a0f93f5d3f34c50a96f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 12:51:28.417921 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80a758-7818-46dd-b87c-0ff9b6675ecc service placement] REQUEST_METHOD: `GET` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 12:51:28.418203 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80a758-7818-46dd-b87c-0ff9b6675ecc service placement] SCRIPT_NAME: `/identity` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 12:51:28.418476 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80a758-7818-46dd-b87c-0ff9b6675ecc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 12:51:28.420270 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.fernet_utils [None req-1e80a758-7818-46dd-b87c-0ff9b6675ecc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 12:51:28.431460 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e80a758-7818-46dd-b87c-0ff9b6675ecc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 12:51:28.433043 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e80a758-7818-46dd-b87c-0ff9b6675ecc service placement] RBAC: Authorization granted {{(pid=81355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 12:51:28.443597 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: [pid: 81355|app: 0|req: 1477/2954] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Oct 6 12:51:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 12:51:38.438515 dv-cinder-856041-2-iscsi devstack@keystone.service[81355]: DEBUG dbcounter [-] [81355] Writing DB stats keystone:SELECT=89 {{(pid=81355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}